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1.  Introduction 


The  United  States  (US)  Army  has  valued  training  across  a  wide  range  of 
performance  areas  and  has  utilized  a  variety  of  platforms  to  instantiate  the  training 
seen  as  best  fit  for  the  knowledge,  skills,  and  abilities  required  for  mission  success. 
As  the  United  States  is  coming  out  of  wartime,  an  opportunity  is  available  to 
consider  the  effectiveness  and  cost  of  training  programs  and  platforms.  In 
particular,  military  training  has  primarily  focused  on  the  principles  of  instructional 
systems  design  (ISD)  as  a  means  to  meet  the  need  of  rapid  training  for  force- 
protection  capability.  Instructional  designers  (IDs)  identify  learning  objectives 
based  on  needs,  which  are  specific  to  a  task  or  mission  and  then  develop 
experiences  that  consist  of  interventions  to  meet  those  learning  objectives.  To  be 
clear,  due  to  the  wartime  needs,  training  has  been  the  emphasis  rather  than  learning. 
Training  is  giving  information  to  a  trainee.  That  trainee  is  not  always  a  learner.  The 
learner  has  to  be  motivated  to  process  and  internalize  the  information  for  actionable 
use.  That  process  is  learning. 

The  present  effort  utilizes  learning  sciences,  which  includes  principles  of  both 
training  and  learning.  The  specific  objective  for  this  effort  addresses  motivation  as 
an  individual  difference  that  can  be  adapted  in  training  to  enhance  learning  rate  and 
retention. 

1.1  Identifying  the  Highest  Impact  Gap 

The  US  Army  Research  Laboratory  (ARL)  is  continuing  to  develop  a  framework 
for  providing  personalized,  on-demand,  computer-based  instruction  using  the 
Generalized  Intelligent  Framework  for  Tutoring  (GIFT;  Sottilare  et  al.  2012; 
Sottilare  2016;  Sottilare  et  al.  2017).  A  variety  of  gaps  exist  pertaining  to  individual 
differences  and  learning  that  might  inform  intelligent  tutoring  systems  (ITSs)  for 
use  in  GIFT  (Sottilare  et  al.  2012;  Goodwin  et  al.  2015).  Increasing  the  level  of 
relevance  and  value  of  this  project  for  GIFT  was  a  top  priority.  ARL,  the  Institute 
for  Simulation  and  Training  (1ST)  at  the  University  of  Central  Florida  (UCF),  and 
The  Boeing  Company  reviewed  a  list  of  26  research  questions,  provided  by  1ST,  to 
identify  a  topic  most  relevant  to  the  needs  of  GIFT.  The  collaborative  meeting 
among  all  parties  narrowed  the  questions  down  to  the  top  3.  The  questions  were 
refined  after  the  discussion  and  the  final  question  addressed  by  this  effort  is:  how 
do  individual  differences  in  motivation  affect  the  learning  rate  and  retention  of  a 
given  learning  task? 

That  question  pertains  to  training  that  increases  time  to  mastery  of  knowledge  and 
skills,  and  retention  in  a  learner.  Motivation  is  an  individual  learning  difference 
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encouraged  through  personalization  of  learning  with  reinforcers  (Graesser  et  al. 
2016).  An  ITS  can  provide  adaptations  at  an  individualized  level  to  support  the 
learner’s  motivation  tendencies,  thus  deemed  an  area  ripe  for  extending  into  GIFT. 
The  present  research  meets  areas  of  need,  identified  for  ARL  by  Goodwin  et  al. 
(2015),  by  providing  more  sufficient  adaptation  for  understanding  the  individual 
learner’s  motivational  traits  and  personalizing  instruction  accordingly.  Therefore, 
addressing  motivation  in  an  adult  learner  fills  a  need  for  ITSs,  which  is  to  have  the 
ability  to  connect  adaptive  training,  education,  and  the  Long-Term  Learner  Model 
(LTLM;  Goodwin  et  al.  2015),  accomplished  by  developing  models  that  allow 
highly  tailored  training  for  increased  learner  readiness. 

1.2  Enhanced  GIFT  via  Phases 

An  enhanced  GIFT  capability  is  being  achieved  through  three  1 -year-long  phases. 
Phase  1  focused  on  the  classification  of  learning  motivation  and  the  evaluation  of 
the  resulting  process.  Phase  2  involves  the  administration  of  a  verification 
experiment.  Phase  3  entails  a  validation  experiment.  The  goal  for  this  report  is  to 
demonstrate  the  process  executed  and  products  delivered  in  Phase  1 . 

2.  Phase  1  Execution 


Phase  1  involved  the  development  of  the  Motivator  Assessment  Tool  (MAT)  to 
enhance  GIFT  by  providing  a  means  to  tailor  motivational  strategies.  The  MAT 
assesses  motivation  traits  of  the  learner  and  environmental  factors  influential  to 
motivation.  Those  variables  are  associated  with  types  of  motivational  reinforcers, 
henceforth  called  motivators.  The  result  from  a  learner  taking  the  MAT  is 
motivators  that  work  best  for  the  life  of  the  learner,  and  thus,  those  individualized 
motivators  can  be  implemented  in  training  to  adapt  the  learning  experience.  First, 
MAT  development  is  essential  because  few  currently  established  adult  motivation 
assessments  exist.  Most  of  the  published  motivation  assessments  are  aimed  at 
school-age  populations  (e.g.,  Harter  1981;  Vallerand  et  al.  1992;  Midgley  et  al. 
2000),  rather  than  adult  populations  (Amabile  et  al.  1994).  Second,  the  MAT  strives 
to  compile  a  comprehensive  picture  of  a  person’s  motivation.  Many  of  the 
established  assessments  only  look  at  one  variable.  Very  few  look  at  multiple 
variables  and  none  of  the  assessments  reviewed  are  as  comprehensive  as  the  MAT. 

To  create  such  an  assessment,  the  first  task  was  to  conduct  an  extensive  literature 
review  regarding  motivation,  individual  differences,  physiological  data,  and 
learning  rate  and  retention.  This  task  also  involved  the  collection  of  motivation 
assessments  and  reinforcement  assessments  currently  being  used  in  education  or 
the  workplace.  For  example,  the  Dunn  Rankin  (Cartwright  and  Cartwright  1970; 
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Landschulz  1978)  is  a  commonly  used  reinforcer  inventory  in  schools  (Landschulz 
1978)  and  the  Work  Preference  Inventory  (Amabile  et  al.  1994)  is  one  used  in 
employment.  In  Task  2,  the  first  iteration  of  the  MAT  and  a  motivator  taxonomy 
were  developed  to  identify  and  define  interrelations  between  motivation  and 
personality  traits,  which  will  later  be  used  to  prescribe  adaptations  within  GIFT  to 
support  the  learner’s  motivation  traits.  Task  3  involved  documenting  outcomes 
from  the  design  of  the  motivational  trait  variable  definition  and  MAT  classifiers 
within  this  report. 

2.1  Task  1:  Data  and  Application  Assessment 

Task  1  began  with  a  literature  review  that  required  scoping  the  macro  and  micro 
issues  pertaining  to  the  topic  of  motivation  and  their  relevance  to  specific  aspects 
of  other  individual  differences  and  the  impact  on  learning.  The  literature  review 
provided  the  foundation  to  construct  the  MAT  and  classifiers  (Appendix  A).  Two 
themes  emerged  from  the  literature  review.  First,  the  topic  of  motivation  is 
immense  and  complex  (Alderman  2013).  Many  of  the  variables  were  classified  as 
separate  entities,  but  had  distinct  parts  of  commonality  among  them.  To  develop  a 
pedagogical  approach  for  providing  adaptations  within  GIFT  to  support  the 
individual  learner’s  motivation,  an  ample  picture  of  an  individual’s  motivation, 
provided  from  an  inclusive  assessment,  encompassing  a  multitude  of  motivational 
variables,  is  required.  This  is  crucial,  because  the  ITS  needs  to  create  a  relationship 
with  the  learner  and  provide  the  perfect  fit.  Motivation  is  inherent  to  a  learner,  and 
thus,  present  prior  to  completing  a  training  session.  However,  motivation  can  be 
enhanced  or  hindered  throughout  the  training  process.  Therefore,  personalization 
within  GIFT  is  necessary  to  deliver  and  maintain  motivation,  adapting  for  optimal 
learning.  This  theme  also  indicated  that  motivators  will  need  to  be  delivered 
according  to  a  schedule  based  on  individual  needs  that  allows  motivation  to  be 
maintained  throughout  a  learning  task. 

The  second  theme  was  that  for  each  variable  included  in  relation  to  motivation,  an 
added  layer  of  interactions  and  complexities  results.  A  reinforcer  is  the  stimulus  in 
the  form  of  a  reward  or  incentive  that  is  received  following  a  desired  behavior  that 
accomplishes  the  goal  (French  1955;  Keller  1987a).  The  term  reinforcer  was 
operationalized  to  motivator  because  it  is  not  reinforcing  a  behavior,  but  proactively 
using  tools  to  increase  learner  motivation.  A  motivator  might  be  more  than  a  reward 
or  incentive,  for  example,  as  a  level  of  autonomy,  frequency,  or  stimulation.  This 
left  the  possibilities  endless.  To  reduce  the  set  of  specific  motivators  for  use  in 
GIFT,  a  list  from  games,  workforce,  and  education  was  compiled  for  the 
comprehensive  list.  The  collaborative  team  discussed  the  list  and  whether  GIFT 
was  capable  of  supporting  each  motivator.  It  was  noted  that  some  motivators  are 


Approved  for  public  release;  distribution  is  unlimited. 

3 


supported  without  an  instructor-like  digital  tokens,  whereas  others  require  an 
instructor,  such  as  food.  An  additional  consideration  for  inclusion  of  the  motivator 
into  the  final  list  was  its  relevance  to  military  training  tasks.  Motivators  that  are 
relevant  to  a  military  task  might  be  a  timer,  leaderboard,  motivational  quotes, 
tokens,  and  feedback.  Motivators  not  relevant  to  a  military  task  include  bubbles, 
playdough,  and  a  sticker. 

This  literature  review  provided  a  goal  for  the  MAT  development,  such  that  the 
MAT  should  enable  properly  individualized  motivators  to  be  determined  from  a 
multitude  of  variables  and  those  motivators  should  then  be  able  to  be  adapted  in  an 
ITS.  The  Assessment  Framework  as  outlined  in  Goodwin  et  al.  (2015)  provides  an 
overview  of  different  types  of  assessments  in  GIFT.  The  MAT  falls  under  the  pre¬ 
training  assessment  that  is  content  independent  based  on  aptitudes  and  traits  with 
macro-adaptations  (Fig.  1).  It  is  envisioned  that  the  learner  will  take  the  MAT  prior 
to  training  with  the  results  being  stored  in  the  LTLM  so  that  the  learner  does  not 
need  to  complete  the  MAT  prior  to  each  training  session.  It  is  based  on  motivational 
traits  and  not  a  person’s  states  in  the  form  of  a  questionnaire.  Since  it  is  not  content 
related,  it  will  affect  the  training  delivery,  but  not  the  content  itself.  However,  it 
should  be  noted  that  the  learner  plan  (explained  in  Task  4)  will  provide  a  pretraining 
assessment  that  is  content  dependent  and  will  be  used  to  adapt  flow  through  content 
based  on  the  learner’s  grasp  of  the  preassessed  content  knowledge.  The  image  in 
Fig.  1  displays  the  assessment  being  used  and  the  timeframe  in  which  it  will  be 
delivered. 


Content 

Dependent 


Pre-Training 

Assessment 


Pre-Assessment 
Phase  2 


In-Training 

Assessment 


Practice  Modifications 
and  Assessment  based 
on  Task 
Phase  2 


Content 
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Motivator  Assessment 
Tool 

Phase  1  Development 


State  Measures- 
Phase  2 


Fig.  1  MAT  placement  in  GIFT  framework 


2.2  Task  2:  Motivator  Assessment  Development 


Task  2  discusses  MAT  development.  Cronbach  and  Meehl’s  (1955)  3  requirements 
for  scale  and  assessment  creation  were  considered  for  validity  and  reliability  in 
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developing  the  MAT:  1)  articulating  a  set  of  theoretical  concepts  and  their 
interrelations,  2)  creating  a  way  to  measure  the  constructs  to  be  evaluated  by  the 
MAT,  and  3)  empirically  testing  (Phase  2)  the  hypothesized  relations  among 
constructs.  This  section  captures  the  process  and  reasoning  behind  the  prototype  of 
MAT.  An  iterative  cycle  occurred  in  developing  the  MAT  such  that  an  initial  MAT 
layout  was  designed  from  the  literature  review.  Based  on  that  design,  criteria  for 
operationalizing  variables  of  motivation  that  impact  corresponding  motivators  were 
determined.  The  resulting  definition  and  scope  led  to  the  creation  of  a  motivator 
taxonomy,  whereby  all  variables  were  documented,  organized,  and  categorized. 
The  taxonomy  fed  back  into  developing  the  MAT,  but  was  also  refined  as  MAT 
development  advanced. 

2.2.1  Motivator  Variable  Definition 

At  the  outset  of  the  project,  motivation  was  originally  thought  of  as  a  state.  Given 
that  the  project  focus  is  to  develop  a  motivator  taxonomy,  which  describes  the 
interaction  of  a  learner’s  motivation  tendencies  over  time  and  targeted  strategies  to 
support  the  learner’s  motivation,  the  term  trait  was  deemed  more  appropriate. 
Learner  motivation  comprises  several  traits,  which  are  variables  that  influence  the 
learner’s  motivation  and  its  resulting  impact  on  learning.  When  the  variables  are 
linked  and  accounted  for,  they  make  up  a  more  complete  description  of  an 
individual’s  motivation.  Variables  that  contribute  to  an  individual’s  motivation 
include  personality  (Goldberg  1981;  Dingman  1990);  grit  (Duckworth  et  al.  2007); 
procrastination  tendencies  (Lay  1986);  intrinsic  versus  extrinsic  motivation  (Ryan 
and  Deci  2000);  task  interest,  autonomy,  or  level  of  learning  control  (Rotter  et  al. 
1972);  self-esteem  (Rosenberg  1965);  self-efficacy  (Bandura  1977);  goal 
orientations  and  tendencies  (Elliot  et  al.  2011);  student  approaches  to  learning 
(SAL;  Bigg  1987);  knowledge  level,  sensitivity  to  punishment  (conflict  resolution), 
and  reward  (Gray  and  McNaughton  2000);  values  (Schwartz  1992);  self-regulation 
(Bandura  1991);  and  learning  styles  (Felder  and  Silverman  1988).  When  the 
motivators  are  provided  to  learners,  they  are  affecting  their  motivational  state. 

Not  all  of  the  variables  are  traits,  some  are  external  influences  that  affect  the 
motivational  state.  GIFT  is  able  to  adapt  based  on  knowledge  level,  which  is  an 
example  of  an  external  influence  of  motivation.  Humans  are  multifaceted; 
therefore,  the  motivation  trait  variable  definition  needs  to  encompass  as  many 
differences  found  that  apply  to  motivation.  Figure  2  depicts  each  of  the  variables 
currently  assumed  as  a  learning  motivation  characteristic. 
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Fig.  2  Motivator  variable  definition  graphic 

2.2.2  Motivator  Assessment  Taxonomy 

The  literature  review  completed  under  Task  1  identified  theoretical  motivation 
constructs  and  variables.  Those  variables  were  organized  into  the  motivation 
taxonomy,  where  interconnections  were  observed.  For  example,  learners  with 
intrinsic  motivation  are  more  likely  to  have  a  high  level  of  conscientiousness  from 
the  Big  5  Personality  model  and  tend  to  have  a  higher  level  of  grit  (Duckworth  et 
al.  2007).  Research  of  these  interconnections  suggested  that  personality  trait 
identification  may  be  relevant  to  individualizing  pedagogy  to  support  the  individual 
learner’s  motivation.  This  means  that  personality  traits  may  be  useful  for 
determining  the  level  of  support  a  learner  requires  and  the  frequency  at  which 
motivators  should  be  presented  (Kyllonen,  Shute  1989;  Busato  et  al.  1999). 
Lameier  et  al.  (2017)  detail  the  interconnectedness  of  pertinent  motivation  variables 
and  the  steps  for  creating  the  MAT.  All  the  background  data  pertaining  to 
motivation  and  individual  differences  described  in  that  paper  provided  the 
foundational  work  needed  for  creating  the  assessment.  The  construction  of  the 
assessment  was  discussed,  but  does  not  provide  a  detailed  description  of  the 
process.  This  report  extends  that  initial  work. 
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A  tailored  pedagogy  is  needed  to  support  the  learner  throughout  the  entire  learning 
experience.  Therefore,  the  MAT  would  serve  that  purpose  by  identifying  a  learner’s 
motivators  for  implementation  throughout  the  learning  experience.  The  first  step 
was  to  compile  existing  assessments  used  for  the  variables  of  personality  and 
motivation  that  pertained  to  learning.  This  led  to  31  assessments  reviewed. 


Only  a  few  assessments,  such  as  The  Motivated  Strategies  for  Learning 
Questionnaire  Manual  (Pintrich  et  al.  1993)  and  the  Adaptive  Patterns  of  Learning 
(Midgley  et  al.  2000),  provide  an  assessment  of  a  broader  scope  of  motivation  by 
addressing  more  than  one  motivation  trait  variable.  To  begin  identifying  the 
interconnections  within  these  various  assessments  and  reduce  the  number  of  overall 
questions,  the  questions  from  each  of  the  assessments  were  typed  up  and 
color-coded.  The  color-coding  was  associated  with  the  assessment  from  which  the 
question  came  and  aided  in  organization  when  the  questions  were  moved  to  reflect 
various  connections.  Figure  3  provides  some  of  the  assessments  and  associated 
questions  that  were  compiled,  their  color-coding,  and  typed  listings. 


GRIT  Sensitivity  to  Punishment  and  sensitivity  to  reward  questionnaire  (GRAY)  RST  Carver/Gray 


Intrinsic 


Perceived 


Motivation  Inventory  (Ryan 


cademic  self-  concept  scale  Liu  and  wangjRotter's  Locus  of  Control  ScaleNeed 


or  cognition  scaleLearning  Self-Regulation  Questionnaire 


_  iBBim 

em  Scale  Revised  study  Process 
Questionnaire  Biggs  Bartle  Test  of  Gamer  Psychology  ARCS  model  questionnaire  RIMMS  PALS  Patterns 
of  adaptive  learning  Reinforcement  Inventories  for  Children  and  Adults  functional  behavior  assessment 
student  reinforcement  survey  forced  choio  reward  preference  questionnaire 

IWDflsI  5ind  iracyv/jHJofj  the  motivators  orientations  questionnaires  Elliot  3x2  goal 
achievement  questionnaire  "o':ra':ed  strategies  for  [earning  questionnaire  (MSLQ) 


|l  do  not  do  assignments  until  just  betore  they  are  to  be  handed  in MfflHBWHBHBBHHmH 
A  letter  may  sit  for  days  after  I  write  it  before  mailing  ^WgSSSBSBSSBBsBKBBSBBSSSi 
Even  with  jobs  that  require  little  else  except  sitting  down  and  doing  them,  I  find  they  seldom  get  done| 


In  preparing  for  some  deadline,  I  often  waste  time  by  doing  otln  i  lliini 
I  always  seem  to  end  up  shopping  for  birthday  or  Christmas  gifts  at  the  last  minute.| 


Fig.  3  Examples  of  the  assessments  compiled  and  associated  questions 
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Similarities  within  the  questions  of  the  assessments  were  documented.  Then,  the 
questions  were  rearranged  into  clusters  of  similarity.  Some  noted  similarities  were 
avoidance,  worry,  giving  up,  boredom,  hard  work,  success,  persistence,  and 
comparing  and  contrasting  to  others.  However,  as  the  questions  were  rearranged, 
connectedness  between  groupings  remained.  The  questions  were  regrouped  until 
the  clusters  were  considered  relatively  distinct.  Personality  types  were  also  taken 
into  consideration  while  grouping  the  questions.  For  example,  if  a  learner  is 
worried,  avoids,  or  has  a  high  stress  level,  then  that  section  of  MAT  questions 
would  possibly  tailor  to  a  person  who  is  high  on  the  neuroticism  scale.  It  may  also 
correlate  with  a  learner  who  is  high  on  the  conscientiousness  scale  because  their 
need  for  detail  and  achievement  could  cause  them  stress.  The  difference  is 
persistence.  Conscientious  learners  are  correlated  with  grit  and  self-efficacy  on  the 
topic,  which  determines  if  they  are  going  to  avoid  or  persevere  based  on 
experiences  of  success  with  the  topic. 

Assessment  items  were  discarded  if  they  would  not  apply  to  the  ITS.  The  following 
are  examples  of  the  questions  that  were  discarded: 

1)  Do  you  often  meet  people  that  you  find  physically  attractive? 

2)  Is  it  easy  for  you  to  associate  taste  and  smells  to  pleasant  events? 

3)  One  of  the  major  reasons  why  we  have  wars  is  because  people  don’t  take 
enough  interest  in  politics. 

4)  There  will  always  be  wars  no  matter  how  hard  people  try  to  prevent  them. 

5)  I  would  run  quickly  if  fire  alarms  in  a  shopping  mall  started  ringing. 

6)  I  would  leave  the  park  if  I  saw  a  group  of  dogs  running  around  barking  at 
people. 

Next,  the  number  of  questions  were  reduced  by  eliminating  redundant  questions. 
Some  questions  were  rewritten  to  make  sense  in  the  context  of  an  individual  student 
learning  within  GIFT.  Finally,  similar  questions  were  collapsed  into  their  basic 
concepts,  which  lowered  the  overall  number  of  questions.  This  first  reduced- 
question  set  provided  approximately  434  questions,  without  inclusion  of  the 
reinforcer  inventory.  Figure  4  shows  a  sample  of  the  questions  from  the  first 
reductions. 
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^  Avoidance/  Stress/ Worry 

1.  I  try  to  avoid  participating  in  class.  (PALS) 

2.  I  avoid  complex  task  because  I  may  not  be  successful  providing  correct  answers,  (complex,  Elliot) 

3.  I  avoid  situations  that  requires  in-depth  thinking,  (need  for  gggpj 

4.  I  avoid  task  in  order  to  not  be  rejected  or  disapproved  by  others,  (sen, to  pun) 

5.  I  work  hard  to  avoid  doing  worse  than  I  normally  do  on  task/exams.  (PALS) 

6.  I  want  to  avoid  becoming  embarrassed  (sen  to  pun) 

7.  I  like  to  avoid  appearing  that  I  am  not  having  trouble  with  the  task  (PALS) 

Fig.  4  First  reduction  questions 

The  resulting  items  were  still  linked  to  the  applicable  motivation  trait  variables. 
After  removing  additional  similar  questions,  the  total  number  of  items  was  reduced 
to  190  questions.  The  categories  were  no  longer  in  order  by  variables  because  of 
their  connectedness.  The  order  became  a  flow  of  the  processes  the  ITS  would  use 
to  tailor  the  learner’s  pedagogy  based  on  item  responses.  Processes  flow  from  “a” 
to  “h”,  whereby  at  each  level,  a  different  categorization  is  identified: 

a)  Identified  interest  in  the  topic, 

b)  Identified  tendencies  for  intrinsic  or  extrinsic  motivation, 

c)  Identified  level  of  effort  the  learner  is  comfortable  extending  for  gaining 
feelings  of  satisfaction, 

d)  Identified  emotional  influences, 

e)  Identified  task  selection  and  goal  orientation, 

f)  Identified  learning  strategy  preferences, 

g)  Identified  learning  environment  preferences,  and 

h)  Identified  reward  orientation  and  motivator  inventory. 

This  flow  became  the  organization  for  the  motivator  taxonomy  (Fig.  5).  The 
taxonomy  enables  a  quick  glance  of  the  key  variables  of  motivation.  The  details 
contained  within  are  a  mapping  of  the  questions  to  the  broader  constructs  associated 
with  motivation. 
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Fig.  5  Image  of  motivation  taxonomy  organization 


The  set  of  190  questions  did  not  address  the  learner’s  task  and  presentation 
preferences,  so  questions  were  created  to  fill  this  gap.  These  additional  questions 
included  items  of  presentation  style  choices,  modality  preference  (e.g.,  visual 
versus  audio  presentation),  and  strategies  needed  to  learn.  Items  pertaining  to 
preferences  on  rewards  were  placed  in  a  reward  orientation  section  that  contained 
a  broader  scope  of  motivators  such  as  preference  for  acknowledgement,  rewards, 
feedback,  and  competition.  The  more  specific  motivators  were  consolidated  in  a 
motivator  inventory.  This  inventory  is  envisioned  to  provide  a  set  of  reward  types 
that  will  be  the  most  beneficial  in  terms  of  supporting  the  learner’s  motivation. 


The  taxonomy  documents  the  interconnections  of  each  item  to  the  personality, 
motivation,  and  demographic  variables  identified  in  prior  research  and  estimates 
from  our  understanding  of  the  research.  The  taxonomy  shown  in  Fig.  6  was  also 
used  to  1)  make  sure  there  was  an  adequate  amount  of  questions  for  each  variable 
and  2)  balance  the  number  of  pro  to  con  statements.  The  taxonomy  is  available 
upon  request. 
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Fig.  6  Pro  and  con  matching 

Each  of  the  remaining  items  were  then  linked  to  the  motivation-trait  variable 
addressed  based  on  the  results  of  the  prior  research  reviewed.  A  coding  legend  was 
created  to  describe  the  connection  to  each  item  and  the  applicable  motivation  trait 
variables,  as  shown  in  Fig.  7. 
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Personality  Variables 

1  it  link  to  someone  with  high  trait,  2  if  linked  to  low  trait,  0  it  no  link 

Grit 

1  if  linked  to  high  grit,  2  if  linked  to  low  grit,  0  if  no  linkage 

Intrinsic  vs.  Extrinsic 

1  if  linked  to  Intrinsic  motivation,  2  if  linked  to  extrinsic  motivation,  0  if  no  link 

Student  Autonomy 

1  if  linked  to  perceived  or  actual  control  of  learning,  2  if  linked  to  lack  of  perceived  or  actual  control  of  learning,  0  if  no  link 

Self-Esteem 

1  if  linked  to  high  self-esteem,  2  if  low  self-esteem,  0  if  no  link 

Self-efficacy 

1  if  linked  to  high  self-efficacy,  2  if  low  self-efficacy,  0  if  no  link 

Fig.  7  Example  of  the  coding  legend 


A  definitions  page  was  also  created  to  establish  the  definitions  of  each  motivation 
trait  variable.  See  Appendix  B  for  definitions  of  the  motivational  trait  variables. 

The  MAT  consisted  of  the  questions  organized  in  the  motivation  taxonomy. 
Directions  were  then  added  to  the  MAT  so  that  it  could  be  administered  in  a  survey 
to  enable  its  validation.  These  directions  took  into  consideration  that  people  do  not 
always  answer  surveys  honestly  (Bandura  1997;  Scandell  et  al.  2000).  The 
directions  clearly  stated  that  this  assessment  is  only  to  be  used  by  the  ITS  to  provide 
the  individual  with  a  motivator  plan  tailored  to  individual  motivation  preferences. 

Scales  were  created  for  the  MAT  survey.  The  collaborative  team  discussed  reasons 
for  different  points  of  scales,  with  5-  and  7-point  scales  being  the  focus.  Research 
in  this  area  is  mixed.  Spagna  (1984)  found  that  without  the  midpoint,  the  results 
had  an  increase  on  the  other  scale  responses.  Dawes  (2001)  found  that  having  a 
midpoint  reduced  the  amount  of  positives.  Garland  (1991)  found  that  having  a 
midpoint  reduced  the  amount  of  negatives.  Many  of  the  motivational  scales 
previously  published  employed  a  5-  or  7-point  scale.  There  are  also  mixed  reviews 
between  5-  and  7-point  scales.  Generally,  the  7-point  scale  is  found  to  have  a  slight 
increase  compared  to  the  5-point  scale  (Diefenbach  et  al.  1993).  Miller  (1956) 
found  that  humans  can  distinguish  7  different  items.  It  was  also  taken  into 
consideration  that  many  individuals  will  not  pick  the  extremes  of  a  scale.  The 
7-point  scale  allows  individuals  to  admit  to  something  without  admitting  to  the 
extreme.  An  even-numbered  scale  was  selected  to  provide  a  forced-choice  scale 
that  eliminates  a  middle  choice,  such  as  neutral  or  not  applicable.  The  scale 
development  was  the  final  piece  in  creating  the  prototype  assessment.  The 
prototype  of  the  MAT  assessment  can  be  seen  in  Appendix  C.  This  prototype  is  a 
large  pool  of  questions  that  will  be  refined  and  reduced  through  a  rigorous 
validation  process. 

The  MAT  will  result  in  identification  of  a  category  of  motivators  best  suited  for 
that  learner.  This  will  occur  by  a  means  of  aggregation  of  scores  that  will  be 
determined  following  validation  of  MAT  items. 
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2.3  Task  3:  Experimentation  in  GIFT  and  Analysis  Plan 


The  first  step  for  validating  the  MAT  was  to  identify  the  correct  assessments  to 
evaluate  against.  The  second  step  was  to  divide  the  assessments  with  the  MAT  into 
3  waves  of  experimentation,  whereby  the  MAT  would  be  reduced  and  refined 
following  each  wave.  The  third  step  was  to  enter  the  MAT  and  the  identified 
assessments  into  GIFT  Cloud  for  administration  to  as  wide  of  a  population  as 
possible  using  both  Amazon  Mechanical  Turk  (MTURK)  and  the  UCF  Sona 
System.  Finally,  an  institutional  review  board  (IRB)  proposal  was  submitted  and 
approved. 

The  first  step  of  reducing  and  refining  the  MAT  was  inserting  it  into  GIFT.  To  do 
that,  all  experimental  questions  were  typed  into  Qualtrics  and  then  uploaded  into 
GIFT  Cloud.  The  MAT  was  broken  down  into  3  sections.  A  pilot  survey  was 
conducted  to  develop  a  sense  of  time  required  for  participants  to  complete.  The 
results  of  the  pilot  survey  showed  that  the  amount  of  questions  in  the  sections  were 
too  long.  The  team  provided  feedback  to  break  the  assessment  down  further  to 
provide  the  participant  short  breaks.  The  MAT  was  broken  into  7  sections:  5 
sections  on  general  motivation  variables  and  2  sections  for  the  reinforcer  inventory. 
Page  numbers  were  also  added  to  let  the  participant  know  the  length  of  all  the 
assessments. 

Wave  1  involves  only  administration  of  the  MAT  (Fig.  8).  This  is  the  first  time  the 
190-item  motivation  assessment  with  the  102-item  motivator  inventory  is  presented 
to  participants  beyond  the  project  researchers.  The  MAT  was  entered  into  GIFT  in 
several  different  ways,  but  due  to  the  sheer  number  of  items,  ultimately  it  was 
broken  into  7  sections,  5  for  the  motivation  variables  and  2  for  the  motivator 
inventory.  The  purpose  for  this  wave  is  to  execute  an  item  analysis  for  internal 
consistency  and  for  face  and  content  validity.  Internal  consistency  is  a  type  of 
reliability  that  addresses  agreement  among  items  within  a  questionnaire  that  are 
intended  to  measure  the  same  construct.  For  Wave  1,  this  answers  the  question,  do 
the  items  that  are  intended  to  reflect  each  variable  (e.g.,  extraversion  or  grit) 
produce  the  same  type  of  response  from  the  participant,  so  that  the  items  “hang” 
together?  Face  validity  is  the  degree  to  which  the  measure  appears  to  measures  what 
it  is  intended  measure.  For  Wave  1,  this  answers  the  question,  does  the  MAT  feel 
like  it  evaluates  motivation?  Content  validity  addresses  the  extent  to  which  a 
measure  includes  all  of  the  facets  of  a  given  construct.  For  Wave  1,  this  answers 
the  question,  does  the  MAT  include  all  the  correct  variables  of  motivation  and 
motivators? 
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Fig.  8  Wave  1  in  GIFT 

Two  hundred  participants  will  be  run  in  Wave  1  and  each  participant  is  expected  to 
take  1  h  to  complete.  After  it  is  administered  and  analyzed,  the  MAT  will  be  refined 
and  reduced.  For  example,  if  all  the  participants  select  neutral,  then  the  question 
would  be  eliminated  or  reworded. 

As  with  Wave  1,  Wave  2  will  be  executed  using  GIFT.  The  MAT  and  selected 
motivation  variable  assessments  are  included  in  this  wave  to  evaluate  content  and 
construct  validity.  Construct  validity  addresses  whether  a  measure  assesses  what  it 
is  intended  to  assess  and  can  be  further  divided  into  convergent  and  discriminant 
validity.  Convergent  validity  refers  to  the  degree  to  which  2  measures  of  constructs 
that  should  relate,  actually  do.  For  example,  in  Wave  2,  items  that  should  be 
measuring  grit  in  the  MAT  strongly  correlate  with  the  grit  and  ambition  short 
questionnaire.  Discriminant  validity  refers  to  the  degree  to  which  2  measures  of 
constructs  that  should  not  relate,  actually  do  not.  For  example,  in  Wave  2,  items 
that  should  be  measuring  grit  in  the  MAT  do  not  strongly  correlate  with  a  need  for 
cognition.  Wave  2  is  also  designed  to  establish  reliability  of  the  MAT.  A  document 
was  created  to  show  the  construct  alpha  correlations.  A  Cronbach’s  alpha 
coefficient  that  is  close  to  0.7  and  above  indicates  the  assessment  items  provide 
acceptable  measures  (Gliem  and  Gliem  2003).  Table  1  presents  the  Cronbach’s 
alphas  for  each  of  the  established  assessments  being  compared  with  the  MAT. 
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Table  1  Cronbach’s  alphas  from  assessments  being  compared  to  MAT 


Motivation 

Intrinsic 

0.74 

Extrinsic  0.62 

Task  value  0.90 

Self-efficacy  0.93 

Control  of  learning  beliefs  0.68 

Motivated  Learning  Strategies 

Learning  strategies,  MLSQ  strategies 

Questionnaire  (MLSQ) 

Self-regulation,  cognitive  metacognitive  rehearsal  -  0.69 

(Pintrich  et  al.  1993) 

Elaboration  0.76 

Organization  0.64 

Critical  thinking  0.80 

Metacognitive  self-regulation  0.79 

Resources  management:  Time  study  0.76 

Effort  regulation  0.69 

Peer  learning  0.76 

Help  seeking  0.52 

Neuroticism  -  0.90 

• 

Anger  0.87 

• 

Anxiety  0.78 

• 

Depression  0.85 

• 

Self-consciousness  0.74 

• 

Impulsiveness  0.72 

• 

Vulnerability  0.76 

Extraversion-  0.89 

• 

Warmth  0.81 

• 

Gregariousness  0.79 

• 

Assertiveness  0.85 

• 

Activity  0.71 

Neo  International  Personality  Item  Pool 
(IPIP)  120 
(Johnson  2014) 

•  Excitement-seeking  0.77 

•  Cheerfulness  0.80 

Openness  to  experience  0.83 

•  Imagination  0.76 

• 

Artistic  interest  0.76 

• 

Emotionality  0.69 

• 

Adventurousness  0.72 

• 

Intellect  0.75 

• 

Liberalism  0.64 

Agreeableness  0.87 

• 

Trust  0.86 

• 

Morality  0.76 

• 

Altruism  0.76 

• 

Cooperation  0.73 

• 

Modesty  0.76 

• 

Sympathy  0.72 
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Table  1  Cronbach’s  alpha  from  assessments  being  compared  to  MAT  (continued) 


Neo  IPIP  120 
(Johnson  2014) 

Conscientiousness  0.90 

•  Self-efficacy  0.63 

•  Orderliness  0.83 

•  Dutifulness  0.69 

•  Achievement  striving  0.80 

•  Self-discipline  0.73 

•  Cautiousness.  0.87 

3x2  achievement 
(Elliot  et  al.  2011) 

Task-approach  goals  0.84 

Task-avoidance  goal  0.80 

Self-approach  goals  0.77 

Other-approach  goals  0.93 

Other-avoidance  goals  0.91 

Carver  White  behavioral  inhibition/ 
avoidance  scale  (BIS/BAS) 
(Carver  and  White  1994; 

Gomez  et  al.  2005) 

BIS  0.74 

BAS  Reward  responsiveness  0.73 

BAS  Drive  0.76 

BAS  Fun  seeking  0.66 

Self-esteem 

(Rosenberg  1965;  Tafarodi  and  Swann 
1995;  Robins  et  al.  2001) 

Alpha  reliabilities  for  the  5-  and  7-point  versions, 
respectively,  were  0.89,  0.93  for  the  RSE;  0.72,  0.79  for  the 
SDE;  and  0.78,  0.78  for  the  IM. 

RR-portrait  value  Schwartz 
(Schwartz  and  Butenko  2014) 

Self-direction  thought  0.72 

Self-direction  action  0.75 

Stimulation  0.82 

Hedonism  0.84 

Achievement  0.86 

Power  dominance  0.88 

Power  resources  0.79 

Face  0.85 

Security  personal  0.71 

Security-societal  0.89 

Conformity-rules  0.94 

Conformity-interpersonal  0.89 

Tradition  0.89 

Humility  0.81 

Benevolence  caring  0.89 

Benevolence  dependability  0.85 

Universalism  tolerance  0.83 

Universalism  concern  0.82 

Universalism  nature  0.92 

Revised  study  process 
(Biggs  et  al.  2001) 

Deep  approach  0.80,  0.70 

Deep  motive  0.62,  0.58 

Deep  strategy  0.63,  0.54 

Surface  approach  0.76,  0.62 

Surface  motive  0.72,  0.42 

Surface  strategy  0.57,  0.53 

Need  for  cognition 
(Sadowski  1993) 

18  items  short 

0.86 

Grit,  ambition 

(Duckwork  and  Quinn  2009) 

0.73-0.83 
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To  note,  originally  the  portrait  value  questionnaire  was  going  to  be  used.  However, 
much  of  the  research  stated  that  it  was  for  children  and  adults  that  did  not  like  to 
think  abstractly.  Most  of  the  research  pointed  to  the  Schwartz  value  survey  but,  it 
could  not  be  found.  So,  Dr  Schwartz  was  contacted  to  find  the  form.  He  said  that 
he  had  not  used  the  survey  in  10  years  and  sent  us  the  new  survey  based  on  a  refined 
and  revised  portrait  value  questionnaire  (Schwartz  et  al.  2016).  This  revision 
expanded  the  original  questionnaire  into  19  values  for  greater  predictive  power  and 
heuristics  (Schwartz  and  Butenko  2014). 

Two  hundred  participants  will  be  run  in  Wave  2  and  analysis  will  be  an  exploratory 
factor  analysis.  A  factor  analysis  processes  the  data  to  determine  correlations  or 
clusters  of  items,  called  factors,  which  will  hopefully  reduce  the  number  of  items 
in  a  measure  (Child  2006).  However,  no  items  will  be  eliminated  prior  to 
completing  Wave  3. 

Wave  3  will  include  the  MAT  and  the  IPIP  (Johnson,  2014)  to  check  reliability 
content  construct  validity.  Approximately  200  participants  will  take  a  variation  of 
the  3  different  courses  for  Wave  2.  Like  Wave  2,  Wave  3  will  be  subjected  to 
exploratory  factor  analysis.  On  the  basis  of  the  results  from  both  Waves  2  and  3, 
the  MAT  will  undergo  a  final  reduction  in  items. 

2.4  Application  of  MAT  in  GIFT 

The  current  flow  of  adaptations  in  GIFT  follows  a  route  from  the  actionable  survey, 
to  the  learner  model,  to  the  pedagogical  model,  then  the  course  adaptation  (Fig.  9). 
The  actionable  survey  is  scored  based  on  the  tags  attached  to  the  concepts  addressed 
by  the  individual  survey  questions.  The  logic  is  written  into  the  survey  for  scoring. 
The  information  collected  from  the  survey  is  sent  to  the  learner  model  (found  in 
advanced  settings)  and  the  scores  for  the  concepts  are  updated.  The  scores  are  then 
used  to  classify  the  learner  as  a  novice,  journeyman,  or  expert  for  each  concept  that 
the  survey  tested.  After  the  concept  is  scored  and  classified,  it  transfers  the 
information  on  the  individual  to  the  pedagogical  model  sets’  metadata  attributes  to 
adapt  the  difficulty  level  of  the  content  based  on  their  level  of  expertise 
classification  for  the  associated  concept.  The  rule  is  created  and  applied  into 
different  parts  of  a  lesson  based  on  the  application  of  Merrill’s  Component  Display 
Theory  (Merrill  1983).  These  rules  are  then  placed  in  the  adaptive  course  flow.  The 
data  are  collected  in  the  Learner  Record  Store  (LRS).  A  learner  profile  that  provides 
students  information  if  they  passed  or  failed  the  course  is  created  by  pulling  the 
data  from  the  LRS. 
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Learner  Configuration 

The  data  passes  through 
learner  configuration. 


Pedagogical  Configuration 

Rules  are  written  for 
personality, 

Rules  are  written  for  type  or 
motivation 

Rules  are  written  for  the 
narrow  scope  such  as 
reinforcers  and  preferences 


Learner  Record  Store 
-  MAT  data  is  stored  for  use 
with  other  tasks 
-points  scored  saved 
Leaderboard  for  each  course 
stored  and  ranked  but  only 
shown  based  on  individual 
motivation 


Actionable  Survey 

-MAT  or  personality 
Tagged  for  scoring 
Possible  multiple  tagging 
Mark  attributes  or  category  to  score 
Scoring  on: 

-  levels  of  personality  and/or 
Stringing  of  variables  for  plans 

ng  of  individual  preference 


The  rules  are  carried  to  the 
adaptive  course  flow  based  on  the 
Merill’s  Component  Display 
Theory. 

-Provide  instructors  specific 
authoring  tool  for  the  rule  and 
adaptions  to  function. 


Learner  Profile 

-Displays  preference 
selections  from  the 
assessment  and  allows  for 
the  learner  to  change 
based  on  their  state. 
Shows  leaderboard  that  is 
saved  in  LRS  and  points. 


Fig.  9  MAT’s  integration  in  GIFT 


For  this  project,  the  MAT  will  be  implemented  as  an  actionable  survey.  Mapping 
the  MAT  into  an  actionable  survey  will  flow  into  this  same  cycle,  but  the  resulting 
adaptations  will  be  designed  to  support  the  learner’s  motivation.  Figure  9  shows 
the  added  pieces  for  the  flow  and  an  explanation  of  the  MAT  into  GIFT. 


2.4.1  Learner  Record  Store 

The  MAT  results  will  need  to  be  stored  in  a  LRS,  which  we  are  assuming  will  be 
the  component  that  stores  the  LTLM.  If  a  learner  takes  a  questionnaire  prior  to  a 
task  every  time,  their  motivation  will  be  depleted.  This  is  consistent  with  the  goal 
of  the  LTLM  to  have  a  historical  learner  model  that  contains  learning  experience 
historical  data,  as  well  as  data  pertaining  to  individual  differences  in  learning  that 
may  be  used  to  tailor  the  pedagogy  and  delivery  mechanisms.  The  learner  should 
need  to  retake  the  assessment  after  a  period  of  time,  such  as  a  year,  or  after  major 
life  events  that  could  jeopardize  the  stability  of  the  trait.  The  person  could  retake  it 
at  any  time  if  they  feel  the  plan  is  not  right  due  to  not  answering  honestly  or  a  major 
life  event.  Some  of  the  information  should  be  shown  to  the  learner  on  the  profile 
where  the  course  history  is  kept.  Learners  should  be  able  to  view  and  verify  the 
specific  preferences  that  have  been  selected  for  them.  They  should  be  able  to 
change  a  preference  if  their  state  is  discordant  with  their  trait  profile  due  to  activity 
just  prior  to  their  learning  session.  For  example,  if  there  is  a  learner  scoreboard,  the 
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data  will  need  to  be  calculated  and  sent  to  the  learner  record  store  where  it 
1)  compiles  other  student’s  points  and  2)  places  the  results,  in  order  of  highest  to 
lowest,  on  the  learner  scoreboard. 

2.4.2  Actionable  Questionnaire  of  MAT 

The  final  state  of  the  MAT  will  be  shaped  by  the  results  from  the  verification 
experiment.  The  assessment  will  then  be  mapped  into  an  actionable  questionnaire. 
Currently,  the  MAT  is  divided  into  sections  based  on  groupings  the  ITS  would  need 
to  know  such  as  intrinsic  motivation,  level  of  effort,  affective  tendencies, 
comparing/competitiveness,  task  (preference  and  strategies),  reward  orientation, 
and  reinforcer  inventory.  Extrinsic  tendencies  will  be  scored  from  the  reverse  of  the 
intrinsic  tendencies.  If  the  results  of  the  MAT  evaluation  show  a  high  correlation 
between  intrinsic  motivation  and  other  motivation  traits  or  even  personality  traits, 
then  it  may  not  be  necessary  to  assess  all  of  the  individual  motivation  traits.  This 
would  greatly  simplify  our  model,  assessment,  and  approach.  However,  if  the 
correlations  are  not  that  strong,  then  it  will  be  important  to  identify  each  of  the 
separate  traits  and  have  logic  for  handling  various  combinations  of  the  traits.  For 
example,  it  is  expected  that  personality  trait  data  may  contribute  to  the  identification 
of  extrinsic  motivation,  but  needs  to  be  confirmed  as  part  of  the  verification 
experiment  data  analysis. 

The  vision  is  that  each  of  these  sections  can  be  scored  to  identify  the  level  or 
existence  of  each  of  the  motivation  traits.  The  scores  for  these  traits  will  be  made 
actionable  by  having  specific  delivery  and  pedagogy  preferences  associated  with 
each  trait,  as  well  as  combinations  of  multiple  traits  since  there  are  expected 
interrelations  among  the  motivation  traits,  personality  traits  and  demographic 
variables.  A  tag  will  be  set  to  score  the  various  sections  from  the  actionable  survey 
(experiment  dependent).  The  assessment  might  be  able  to  score  on  multiple  levels, 
such  as  a  general  level  that  would  classify  a  learner  as  having  intrinsic  or  extrinsic 
motivation,  as  well  as  narrow  levels,  such  as  classifying  a  learner  as  having  an 
introverted  personality  trait  and  extrinsic  motivation.  This  may  call  for  a  different 
way  of  scoring  the  assessment  and  a  different  tagging  system.  Current  capabilities 
of  GIFT  allow  for  one  tag  to  an  attribute  per  question.  For  different  types  of 
learners,  certain  questions  may  be  used  for  more  than  one  score.  For  example,  Tag 
1  would  be  scored  with  the  Intrinsic  Fearner  Plan  such  that  an  intrinsic  learner’s 
correlates  are  with  a  set  of  variables  that  need  to  be  scored  throughout  the 
assessment  and  not  just  based  on  a  few  questions  measuring  one  attribute.  Tag  2 
might  be  with  the  Extrinsic  Fearner  Plan  such  that  the  extrinsic  learner  will  need  to 
provide  the  personality  type.  Tag  3  might  be  with  Personality  Type,  which  is  needed 
to  further  guide  the  extrinsic  learner  plan  to  accurately  provide  the  type  of  schedule, 
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level  of  support  needed,  and  so  forth  (yet  to  be  determined  based  upon  the 
Wave  1-3  results).  A  separate  tag  might  not  be  needed  if  the  item  is  specific  to  only 
intrinsic  or  extrinsic  learners.  Therefore,  the  MAT  will  be  tailored  to  motivation 
such  as  a  type  of  general  plan  down  to  the  more  narrow  strategies  and  preferences 
a  learner  needs  for  them  to  learn  optimally.  For  the  ITS  to  have  a  complete  picture, 
it  needs  to  be  provided  as  various  levels  of  information. 

Based  upon  the  results  from  Waves  1-3,  a  different  path  for  mapping  individual 
differences  into  training  and  strategies  might  occur.  Rules  might  be  written  for 
different  sections  of  the  assessment  to  provide  different  things  for  the  learner 
(Table  2).  The  comments  are  anticipated  to  begin  tagging  people  based  on  what  is 
known  from  personality  or  intrinsic/extrinsic  learners.  The  rule  is  the  way  in  which 
a  tag  might  be  implemented  in  a  GIFT  course. 

Table  2  Example  of  rules  from  MAT 


Section  Statement 


I  like  to  delay  my 
Effort  1  work,  until  it  is 
necessary. 


Comments 


Delaying  work  is  associated  with 
specific  personalities  such  as  ties 
to  neuroticism  (Watson  2001)  if 
avoiding  the  task  because  they  do 
not  feel  they  will  be  successful.  A 
person  who  is  conscientious 
should  score  reversed  on  this 
question  because  of  the  negative 
link  to  procrastination  found  in 
research.  Low  conscientiousness 
might  need  the  reminders  to  begin 
because  of  the  level  of  focus  and 
impulsiveness. 


Rule 


If  the  individual  scores  higher 
(most  individuals  will  not 
honestly  pick  the  extreme)  on 
this  question: 

Rule  for  this  learner:  Motivation 
will  begin  prior  to  the  task. 

Such  as  a  quote,  music,  physical 
activity,  or  animation. 

(reinforcer  inventory) 

Praise  can  also  be  provided  to 
the  learner  in  the  form  of 
supportive  empathetic 
reminders,  (reward  orientation 
and  reinforcer  inventory) 


I  need  to  feel  in 
Intrinsic  complete  control, 
over  my  learning. 


This  is  intended  for  scoring  an 
intrinsic  learner  plan  seen  below. 
However,  intrinsic  tendencies  are 
also  found  in  personalities  such  as 
openness.  (Entwistle  1988) 


If  a  person  scores  high  in  the 
intrinsic  or  personality  then  they 
will  be  able  to  make  decisions  to 
change  the  pathways  of  their 
learning  as  seen  in  the  intrinsic 
learner  plan. 

Rule:  Apply  plan  with 
maximum  autonomy  and  choice. 


This  is  an  extrinsic  learner  so  the 
score  would  be  factored  into  the 
different  plans. 


Added  points  for  starting  the 
task  on  time  could  be  provided 
to  the  learner  to  increase  the 
chance  of  starting  right  away  on 
other  tasks,  (reinforcer 
inventory) 
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Table  2  Example  of  rules  from  MAT  (continued) 


Section  Statement  Comments 


Rule 


Effort  2 


I  can  pass  by 
memorizing  the 
information,  even 
if  I  do  not 
understand  the 
material. 


This  person  is  a  strategic  and 
extrinsic  learner.  For 
personality  they  give  up  easily 
or  have  had  prior  success 
toward  this  type  of  learning. 

If  it  is  extrinsic,  assumptions 
could  be  made  on  the  type  of 
personality  this  would  tie  to. 

A  person  who  has  a  fear  of 
seeking  help,  possibly  an 
introvert.  Doesn’t  want  to  look 
stupid  to  peers  and  avoid 
failure  such  as  a  neuroticism. 
Or  someone  who  has  a  low 
level  of  patience  and  focus 
such  as  low  conscientiousness. 


This  person  would  need  to  begin 
the  course  at  a  lower  complexity 
to  ensure  success  and  being 
proactive  on  limiting  their 
chances  on  not  understanding. 

This  type  of  person  needs  more 
examples,  scenarios,  and/or 
forced  reflections  with  the 
content.  Providing  more 
examples  forces  them  past  the 
memorization. 

Questions  at  a  higher  level  of 
Bloom’s  would  also  allow  the 
ITS  to  see  that  the  student 
cannot  apply  the  information  but 
just  spit  it  out. 


This  person  needs  a  supportive 
plan. 


Emotional 
Influences  1 


They  need  to  begin  at  a  lower 
complexity  and  move  to  a 
higher. 


Thinking  about 
mistakes  makes 
me  worry. 


Ties  to  low  self-esteem,  low 
self-efficacy,  neuroticism, 
possibly  high  conscientious¬ 
ness,  strategic  learner,  and. 
Sensitivity  to  Punishment. 


The  practice  level  needs  to 
begin  at  the  demonstration  level, 
guidance,  and  then  individual. 
When  getting  a  wrong  answer 
the  feedback  needs  to  be 
supportive  and  empathetic  with 
a  quote  about  growth  mind  set 
on  the  mistake. 

Example:  I  am  glad  you  made 
this  mistake.  Remember  it  helps 
you  grow.  You  are  making 
progress  in  learning!  Let  me 
demonstrate  and  explain  this 
question.  Now  you  correct  your 
mistake  to  prove  to  yourself  you 
learned. 


After  they  rework  the  question: 
Excellent  accomplishment.  Here 
is  a  bonus  200  for  progressing 
further  toward  your  goal  with 
persistence. 
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Table  2  Example  of  rules  from  MAT  (continued) 


Section 

Statement 

Comments 

Rule  Example 

Ranking  1 

I  like  to  compare 
myself  to  others  in  an 
anonymous  manner. 

This  individual  has  ties  to 
introverts.  If  they  like  to 
compare  to  others  the 
question  has  ties  to  extrinsic. 
They  are  sensitive  to  their 
social  image. 

Rule:  Show  leaderboard 
of  points  with  a  fake 
name. 

Ranking  3 

Being  compared  to 
others  is  something  I 
detest. 

This  individual  also  has  ties 
to  an  introvert  and 
neuroticism  but  at  a  different 
level  from  the  question  above. 
The  impact  of  having 
comparisons  could  cause  this 
learner  to  shut  down. 

Do  not  rank  show  the 
individual  the 
leaderboard.  No 
competitions  on  a  task  or 
assessment. 

Task/Strategy 

I  prefer  to  write  my 
own  goals. 

This  is  only  given  to  an 
intrinsically  motivated  person 
because  they  are  already  deep 
level  learners  and  strive  for 
maximum  understanding. 
Openness  is  associated  with 
intrinsic  learners. 

Rule:  Provide  the  learner 
the  option  to  write  their 
own  goal  if  an  intrinsic 
learner  or  the  personality 
that  correlates  with 
intrinsic  learning. 

Task/Strategy  2 

Helping  me  manage 
my  time  will  reduce 
“off  task”  behavior. 

This  would  help  impulsive 
learners  that  have  a  difficult 
time  staying  on  task.  Special 
education  teachers  provide 
buzzers  to  help  learners  self- 
regulate  their  time  off  a 
task. 

Rule:  A  time  schedule  to 
set  reminders  for  the 
learner.  This  could  be  a 
short  brain  break  with 
physical  movement  re¬ 
set  the  mind. 

This  question  will  also 
related  to  brain  breaks. 
Learners  who  are  more 
impulsive  benefit  from  more 
frequent  brain  breaks. 

Reward 
Orientation  1 

My  satisfaction 
increases,  when  I 
receive  points  and 
feedback  on  my 
accomplishments. 

Extrinsic  learner  plan. 
Personality  would  be 
dependent  on  schedule 
example  neuroticism  would 
need  this  more  frequently. 

Rule:  Provide  points 
and  feedback  on 
accomplishments  based 
on  personality  schedule. 

Neuroticism:  Provide 
after  every  item  for 
points  and  feedback 
after  every  3-5. 

Conscientiousness: 

After  a  successful 
completion  of  a  subtask 
in  the  course. 

Reward 
Orientation  2 

I  would  rather  not 
receive  a  reward. 

A  intrinsic  learner  would 
not  care  if  they  received  a 
reward.  Accomplishment  is 
their  reward. 

A  surprise  reward 
unrelated  to  the  task  but 
not  every  task. 
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2.4.3  Course  Adaptations 

The  rules  created  from  the  MAT  might  be  sent  to  the  various  parts  of  the  Merrill’s 
(1983)  component  display  module  for  adapting  the  course.  The  course  itself  will 
need  new  tools  to  provide  the  author.  Further  discussion  on  the  course  adaptations 
provided  follow. 

3.  Task  4:  Design  Ideas  for  Phase  2 

Manipulation  of  instructional  strategies  targeted  to  optimize  the  motivation  traits 
described  previously  might  influence  the  learning  rate  and  retention  of  a  task.  The 
complexity  of  the  design  for  providing  an  instructional  strategy  within  GIFT  that  is 
tailored  to  these  individual  differences  requires  extensive  and  fine-grained 
planning.  To  measure  effects  between  the  different  tailored  instructional  strategies 
requires  a  control  version,  which  does  not  have  any  tailored  strategies.  The  first 
instructional  strategy  variable  to  be  tested  is  the  level  of  control  provided  to  the 
learner  and  the  resulting  autonomy  provided  to  the  learner.  Based  on  the  prior 
sections,  the  optimal  level  of  control,  perceived  autonomy,  and  motivator 
preference  is  based  on  the  learner’s  motivation  type  (intrinsic  vs.  extrinsic), 
achievement  goal  disposition,  affect,  general  self-efficacy,  and  personality.  This 
variable  will  have  2  manipulations  in  our  planned  experiment:  one  in  which  the 
control  level  is  appropriate  given  the  learner’s  MAT  results  and  a  yoked 
manipulation  in  which  the  level  of  control  is  discordant  with  the  learner’s  MAT 
results  (yoked  condition).  The  following  sections  describe  initial  ideas  for  the 
design  of  learning  strategies.  These  ideas  are  contingent  upon  the  MAT  validation 
results  from  the  waves. 

3.1  Background  for  the  Learner  Plans  with  Personality  Traits 

Research  has  provided  evidence  that  some  personality  traits  are  associate  with 
increased  academic  motivation  (Komarraju  et  al.  2009).  The  following  discussion 
on  personality  is  based  on  the  use  of  the  Big  5  Personality  model  described 
previously,  with  the  traits  of  extraversion,  neuroticism,  openness,  agreeableness 
and  conscientiousness.  This  comparison  builds  an  extension  of  motivation.  The 
personality  traits  of  conscientiousness,  extraversion,  and  openness  appear  to  have 
the  strongest  goal  orientation  (Payne  et  al.  2007).  Individuals  who  have  high 
neuroticism  and  low  extraversion  are  more  prone  to  fear  of  failure  and  avoidance 
(Payne  et  al.  2007).  Neuroticism  is  negatively  related  to  the  theories  of  goal  setting, 
expectancy  theory,  and  self-efficacy  theory,  where  conscientiousness  was 
positively  related  to  all  3  (Judge  and  Ilies  2002).  Therefore,  the  optimal  level  of 
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support  and  timing  for  an  individual  may  be  dependent  on  their  personality  trait 
composition. 

For  example,  the  conscientious  learner  is  driven  by  achievement;  therefore, 
motivation  does  not  need  to  occur  from  the  very  beginning  of  the  course.  Given  this 
drive  for  achievement,  it  is  important  that  they  are  provided  with  feedback  for 
correct  and  incorrect  answers.  Since  they  are  sensitive  to  punishment,  feedback  for 
incorrect  answers  will  aim  toward  learning  and  not  be  administered  in  a  punitive 
manner.  Conscientious  learners  also  prefer  feedback  regarding  their  long-term 
effort.  Since  conscientious  learners  value  achievement,  they  can  handle  a  more 
challenging  task  and  persist  for  a  longer  period  of  time.  Therefore,  if  the 
conscientious  learner  is  provided  an  external  motivator,  its  recommended 
implementation  is  to  stretch  it  out  over  time.  For  example,  tokens  would  be 
provided  in  larger  quantities  over  a  longer  period.  If  the  MAT  validation  waves 
demonstrate  that  intrinsic  motivation  is  related  to  high  conscientiousness  and 
openness,  but  low  neuroticism,  then  this  would  indicate  that  they  do  not  need 
motivators. 

3.1.1  Conscientiousness  and  Motivation 

Learners  that  score  low  in  conscientiousness  are  more  likely  to  procrastinate  (Steel 
2007).  They  are  linked  to  impulsivity  and  carelessness,  impatience,  and  distraction 
(Cheramie  and  Simmering  2010).  Cheramie  and  Simmering  (2010)  found  low 
conscientious  learners  need  higher  accountability  for  higher  learning  outcomes. 
These  can  be  stimulated  to  higher  levels  of  learning.  Learners  need  to  be  motivated 
from  the  beginning,  even  before  the  lesson  begins.  The  feedback  needs  to  be  more 
tailored  to  staying  on  task  and  effort.  This  type  of  learner  needs  to  play  a  more 
active  role  in  learning.  They  need  to  be  constructing  and  not  just  soaking  in  the 
information. 

3.1.2  Openness  and  Motivation 

Learners  with  high  openness  tend  to  have  more  interest,  creativity,  imagination,  and 
more  intrinsic  motivation  (Entwistle  1988).  They  have  a  tendency  to  be  deep 
learners  (Swanberg  and  Martinsen  2010).  They  positively  correlate  with 
universalism  (equality,  social  justice,  protecting  the  environment),  self-direction 
(independence),  hedonism  (fun/enjoyment),  benevolence  (welfare  of  people  close 
to  them),  and  stimulation  values.  They  are  negatively  correlated  to  conformity, 
security,  and  tradition  and  the  most  negative  out  of  all  the  personalities  to  value  of 
power  (Parks-Leduc  et  al.  2015).  They  are  negatively  related  to  sensitivity  of 
punishment  (Mitchell  et  al.  2007).  They  are  innovators  that  are  seen  as  investigative 
and  artistic  (McCrae  2000).  On  the  vocational  interest,  they  are  correlated  with 
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investigative,  social,  and  artistic  interest  (Mount  et  al.;  Rounds  2005)  and  positively 
associated  with  all  6  self-efficacy  types  in  interest  (Hartman  and  Bentz  2007).  They 
prefer  open  assessments  that  are  not  analytic,  concise,  or  multiple  choice 
(Chamorro-Premuzic  et  al.  2005).  An  open  learner  is  linked  to  autonomy  and  the 
need  to  express  their  creativity.  Motivation  is  does  not  need  to  be  supported  at  the 
start  of  the  learning  process,  but  one  needs  to  be  careful  to  foster  their  level  for 
intrinsic  learning. 

On  the  flipside,  low  openness  is  related  to  traits  such  as  less  abstract  thinking,  low 
intrinsic  motivation,  less  willing  to  change,  and  high  sensitivity  to  punishment 
because  of  sensitivity  to  cues  of  novelty  and  punishment. 

3.1.3  Extraversion  and  Motivation 

Extraverted  learners  need  a  more  social  and  interactive  environment.  They  are 
motivated  to  achieve  rewards  with  goals.  They  have  a  high  performance  and  are 
connected  to  relationship,  adventure,  achievement,  being  agreeable,  escapism,  and 
competence.  They  like  challenges.  However,  they  can  reach  cognitive  decisions 
prematurely  (Matthews  1997)  and  can  talk  more  than  studying,  which  can  have  a 
negative  impact  on  academic  performance  (O’Connor  and  Paunonen  2007).  They 
correlate  with  achievement,  self-direction,  power,  stimulation,  and  hedonism. 
Extraverts  tend  to  be  happier  and  have  a  higher  well-being  in  general  (Anglim  and 
Grant  2016;  Steel  et  al.  2008).  They  have  a  tendency  to  cope  with  stress  because  of 
their  level  of  stimulation  and  need  for  challenge  (Carver  and  Conner- Smith  2010), 
which  translates  to  possibly  higher  self-efficacy  in  general  (Ebstrup  et  al.  2011; 
§ahin  and  Cctin  2017).  Extraversion  correlates  with  social  interest  versus 
vocational  interest,  and  relates  to  artistic,  social,  and  enterprising  self-efficacy 
(Holland  1997).  They  are  deep  and  strategic  learners.  They  correlate  with  mastery 
approach  and  avoidance  (Chen  and  Zhang  2011;  McCabe  et  al.  2013).  Their  grade 
is  dependent  on  the  type  of  assessment.  They  prefer  oral  exams,  short  multiple 
choice,  and  group  work  (Chamorro-Premuzic  et  al.  2005).  They  might  need  more 
videos  in  their  learning  and  opportunity  to  share  their  thoughts,  as  well  as 
complexity  added  to  their  learning. 

The  opposite  is  an  introvert.  This  learner  prefers  lower  stimulation.  Lower  arousal 
tends  to  enable  higher  performance.  They  are  reserved,  cautious,  sensitive  to 
punishment,  and  have  a  high  sensitivity  to  rewards  (Corr  2004).  They  are  prone  to 
fear  of  failure  and  avoidance  (Payne  et  al.  2007). 

3.1.4  Neuroticism  and  Motivation 

Neuroticism  students  tend  to  be  lower  in  openness  and  conscientiousness 
(Chamorro-Premuzic  et  al.  2005).  They  have  higher  anxiety,  anger,  depression, 

Approved  for  public  release;  distribution  is  unlimited. 

24 


self-consciousness,  impulsiveness,  and  vulnerability  (Komarraju  et  al.  2011). 
Correlations  for  well-being  are  negative  to  neuroticism  perhaps  because  they  are 
predisposed  to  experience  negative  affect  (Costa  and  McCrae,  1992;  Anglim  and 
Grant  2016).  Neurotic  learners  are  more  failure-oriented  and  have  a  higher  degree 
of  external  motivations  and  poorer  achievement  (Komarraju  et  al.  2011).  They 
withdraw  and  tend  to  produce  more  amotivated  individuals.  They  are  more  emotion 
focused  and  less  focused  on  the  task.  There  is  evidence  that  they  do  better  when 
kept  busy  and  poor  performance  occurs  when  not.  They  are  linked  to  lower 
self-esteem  and  liking  short-term  rewards.  They  are  more  surface-level  learners 
because  of  their  fear  of  failure  (Busato  et  al.  1998).  They  do  not  have  associations 
with  value.  They  are  correlated  with  sensitivity  to  punishment  because  they  are 
vulnerable  and  had  low  self-esteem  (Mitchell  et  al.  2007).  They  are  more  sensitive 
to  ridicule,  are  uncomfortable  around  others,  exhibit  shy  behavior  and  poor  coping 
skills  in  stressful  environments,  and  can  be  dependent  on  others  (Costa  and  McCrae 
1992).  This  learner  would  not  fare  well  in  competing  or  comparing  to  others.  They 
have  ties  to  procrastination  (Watson  2001)  and  a  tendency  toward  being  negative 
(Matthews  and  Zeidner  2004).  They  prefer  assessments  that  are  not  continuous. 
Four  facets  of  neuroticism  are  related  to  performance  approach  goals,  5  facets  of 
mastery  avoidance,  and  all  6  facets  were  related  to  performance  avoidance  goals 
(McCabe  et  al.  2013).  Research  has  shown  ways  to  help  avoidance  individuals  by 
reducing  negative  consequences,  removing  distractors  such  as  dual  tasks,  not 
providing  time  pressures,  providing  structure  and  focus,  and  allowing  them  to  clear 
their  mind  regularly  (Roskes  et  al.  2014).  On  the  flipside,  the  learner  is  emotionally 
stable.  Low  neuroticism  is  associated  with  preferring  essays  or  oral  exams 
(Chamorro-Premuzic  and  Furnham  2003). 

3.1.5  Agreeableness  and  Motivation 

Agreeableness  is  the  trait  that  is  found  to  least  impact  academic  achievement. 
(Chamorro-Premuzic  and  Fumham  2003).  Learners  high  in  agreeableness  are 
compliant,  put  others  before  themselves,  and  are  trusting  and  good  natured. 
Agreeableness  is  positively  correlated  with  the  values  of  benevolence,  tradition, 
and  conformity,  and  is  negatively  correlated  with  power  and  achievement  because 
of  putting  others  first,  hedonism,  stimulation,  and  self-direction.  Individuals  with 
high  agreeableness  avoid  punishment  due  to  their  sensitivity  to  punishment  and 
desire  to  not  displease  another  or  be  perceived  as  disagreeable  (Mitchell  et  al. 
2007). 

On  the  flipside,  individuals  low  in  agreeableness  are  manipulative  and  competitive. 
They  are  high  on  the  sensitivity  to  rewards  from  increased  aggressiveness  and 
motivation  to  obtain  rewards.  They  will  pursue  a  reward  even  at  the  expense  of 
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others.  Consequently,  learners  with  low  agreeableness  require  more  checks  on 
learning  during  instruction  and  immediate  feedback  on  the  correct  way  of 
performing  a  task.  They  may  benefit  from  tasks  that  engage  them  in  the  learning 
activity.  Low  agreeableness  learners  need  more  active  learning  versus  passive 
learning.  Agreeableness  has  the  least  influence  on  academic  achievement  of  all  of 
the  Big  5  Personality  traits.  However,  agreeableness  is  an  important  trait  for 
successful  team  training  outcomes. 

3.1.6  Summary  of  Personality  Traits 

An  individual  contains  portions  of  all  5  traits  to  varying  degrees.  When  considering 
the  best  approach  to  motivate  an  individual,  their  level  for  all  5  traits  needs  to  be 
considered.  The  traits  will  interplay.  A  learner  that  is  high  in  openness, 
conscientiousness,  and  neuroticism  has  drive  and  persistence,  but  emotional 
instability.  An  individual  with  low  openness  and  conscientiousness,  and  high 
neuroticism  needs  a  different  type  of  support.  Thus,  the  present  effort  mapped  the 
personality  traits  to  different  learning  strategies  to  take  into  consideration  all  the 
traits  a  person  may  bring  to  the  table  when  learning.  They  are  broken  into  4 
quadrants  (Fig.  10).  The  main  traits  considered  are  conscientiousness,  openness, 
and  neuroticism  because  of  the  connection  to  academic  performance  (Chamorro- 
Premuzic  and  Furnham  2003;  Paunonen  and  Ashton  2001).  There  is  also  research 
pointing  to  linkage  between  academic  performance  and  extraversion,  and  to  a  lesser 
degree,  agreeableness.  High  extraversion  is  correlated  with  higher  stimulation  that 
is  brought  forth  in  social  situations.  Whereas,  introverts  are  over  stimulated  in 
social  situations  (Eysenck  and  Eysenck  1985).  Introverts  are  also  prone  to  being 
avoidant.  Perhaps  this  might  be  because  of  a  lack  of  coping  with  overstimulation 
situations.  However,  in  GIFT’S  current  state,  overstimulation  from  social  situation 
is  not  a  motivational  factor  for  the  introvert  because  of  one-on-one  instruction  from 
the  tutor.  Figure  10  gives  a  broad  overview  of  the  4  quadrants  that  will  be  looked 
at  more  in  depth.  Personality  is  placed  within  the  quadrants  and  variables  are  folded 
within.  This  produced  strategies  and  motivators  related  to  the  individual  learner’s 
needs  (dependent  on  experiment  of  MAT)  because  the  level  of  support  and 
frequency  matches  motivators. 
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Quadrant  1 

Intrinsic  Tendencies 

Quadrant  2 

High  Neuroticism 

Quadrant  3 

Low  conscientious  or  openness 

Quadrant  4 

Low  conscientiousness  and/or  openness 
and /  or  high  neuroticism 

Fig.  10  Four  quadrants  macro  overview 

The  design  suggested  herein  will  be  modified  according  to  data  from  the  validation 
waves,  but  will  serve  as  a  plan  for  executing  adaptation  within  GIFT.  First,  the 
MAT  determines  an  individual’s  intrinsic  and  extrinsic  motivation  tendencies. 
Intrinsic  motivation  is  something  of  inherent  satisfaction  (Calder  and  Staw  1975; 
Porac  and  Meindl  1982)  and  extrinsic  motivation  leads  to  doing  something  because 
of  the  outcome  it  leads  (Deci  and  Cascio  1972).  These  quadrants  can  be  mapped  by 
personality  or  stringing  of  variables  from  the  MAT.  Quadrant  1  is  an  intrinsic 
motivated  learner.  Quadrants  2-4  are  designated  for  various  levels  of  extrinsic 
motivated  learners.  Ryan  and  Deci  (2000)  found  that  there  is  a  continuum  of 
determination  and  control  ranging  from  incongruence  with  one’s  values  and  needs 
(integrated)  to  complete  extrinsic  rewards  or  punishments  (external)  and  is 
discussed  in  detail  in  the  extrinsic  quadrants. 

3.2  Quadrant  1:  Intrinsic  Learner  Plan 

The  reason  for  an  intrinsic  learner  plan  is  present  in  research  on  learning.  This 
section  is  based  on  quadrant  1  and  details  the  reason  why  GIFT  needs  to  account 
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for  this  type  of  learner  when  adapting.  Learners  with  intrinsic  motivation  are 
internally  driven  to  succeed  in  the  acquisition  of  knowledge,  achievement,  and 
stimulation.  They  are  deep-level  learners,  high  in  persistence,  and  have  a  positive 
self-esteem  (Deci  and  Ryan  1995),  which  assists  in  the  retention  of  knowledge. 
Learners  with  intrinsic  motivation  are  empowered  through  relevance  (Baumeister 
and  Leary  1995),  autonomy  over  their  learning  (deCharms  1968),  and  competence 
(Harter  1978;  White  1963).  Literature  presents  that  applying  external  forces  can 
suppress  intrinsic  motivation  (Deci  et  al.  1999).  These  external  pressures  reduce  the 
internal  locus  of  control,  thereby  reducing  natural  intrinsic  motivation.  Traditional 
teaching  practices  and  many  ITS  design  models  place  external  limitations,  rewards, 
and  control  that  are  undesirable  and  a  hindrance  to  learners  with  intrinsic 
motivation  that  reduce  performance.  Including  pedagogy  consistent  with  the 
preferences  of  the  learner  with  intrinsic  motivation  ensures  that  the  learning 
environment  nurtures  their  internal  drive  rather  than  detracting  it. 

The  self-determination  theory  (SDT)  looks  at  the  broad  picture  of  a  learner  and 
their  natural,  intrinsic  drive  for  learning.  While  the  Attention,  Relevance, 
Confidence,  and  Satisfaction  (ARCS)  model  of  motivation  (Keller  1987b)  provides 
a  categorization  of  instructional  motivation  and  instructional  strategies,  application 
of  the  ARCS  model  does  not  alter  individual  behavior  or  teach  self-motivation 
(Keller  1987b).  The  intrinsic  motivated  learner  is  already  high  in  the  4  ARCS 
variables  without  requiring  external  incentives  or  controls,  but  the  strategies 
provide  considerations  on  instructional  motivation  for  an  intrinsic  motivated 
learner  in  GIFT.  Attentional  strategies  provide  variance  on  individual  learners,  such 
as  varying  tone  of  voice,  medium  (video,  sound),  and  style  (humor,  pace,  and  noise 
level).  Relevance  during  instruction  is  developed  through  challenging  goals  and 
relation  to  the  future.  This  matches  an  intrinsic  motivated  learner’s  quest  of 
knowledge.  Confidence  is  related  to  affect,  self-esteem,  and  self-efficacy,  where 
intrinsic  motivated  learners  are  prone  be  attributed  to  positive  outlook  from  a  deep 
level  of  learning  values.  Confidence  is  enhanced  from  a  self-plan  of  work,  low  risk 
for  new  skills,  and  independence.  Intrinsic  motivated  learners  need  to  have  the 
strings  cut  from  traditional  learning  to  allow  a  more  natural  satisfaction  of 
understanding.  Of  course,  it  never  hurts  to  surprise  someone  by  throwing  a 
noncontingent  reward  that  enhances,  but  does  not  undermines  values. 

In  addition  to  providing  support  for  the  ARCS  model  variables,  the  ITS  in  GIFT 
needs  to  also  provide  a  means  of  supporting  the  intrinsic  motivated  learner’s  self¬ 
regulation  strategies.  Self-regulated  learning  (SRL)  and  a  learning  strategy  that 
supports  the  learner  with  intrinsic  motivation  are  similar  in  terms  of  the  level  of 
choice,  voice,  and  relevance  provided  throughout  the  learning  process  (Azevedo 
2005).  SRL  is  the  initiation  of  one’s  thoughts,  feelings,  and  behaviors  toward 
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reaching  a  goal  (Zimmerman  2001;  Zimmerman  and  Schunk  2011).  Learners  with 
intrinsic  motivation  have  a  higher  proper  use  of  metacognitive  and  cognitive 
strategies  like  elaboration,  rehearsal,  and  organization  (Wolters  1998).  However, 
choices  for  length,  amount,  and  type  of  organization  options  can  be  provided  during 
the  task  and  practice.  Other  self-regulated  strategies  to  be  considered  are  managing 
time  effectively,  actively  seeking  help,  believing  in  one’s  self,  and  conducting  self- 
evaluations  (Schunk  and  Ertmer  2000).  Intrinsic  motivated  learners  fit  the  profile 
for  mastery  goal  learners.  They  strive  to  increase  competence  of  themselves  and 
reach  higher  levels  of  learning,  knowledge,  and  performance  (Bell  and  Kozlowiski 
2002),  and  higher  motivation  (Colquitt  and  Simmering  1998).  The  Universal 
Design  for  Learning  is  a  framework  for  designing  lessons  based  off  the  notion 
everyone  has  different  learner  variables.  The  needs  of  the  variables  are 
accomplished  through  different  “pathways”  ranging  from  tools,  strategies,  and 
scaffolds  for  reaching  mastery  or  expertise  (Meyer  et  al.  2014).  It  provides 
guidelines  on  when  to  implement  different  ways  of  personalizing.  The  intrinsic 
motivated  learner  should  have  options  for  different  ways  to  display  the  information, 
such  as  auditory  or  visual  information.  GIFT  is  able  to  provide  clarification  of 
vocabulary  and  symbols,  illustrate  through  multiple  media  types,  activate  or 
provide  background  knowledge,  highlight  patterns,  and  describe  big  ideas  and  their 
relationships.  The  intrinsic  motivated  learner  also  needs  to  be  provided  different 
ways  of  action  and  expression. 

Many  researchers  have  tested  different  pieces  of  self-regulation  and  extrinsic 
effects  with  intrinsic  motivated  learners  with  ITS  or  computer-based  tutoring 
(CBT).  However,  unless  we  account  for  all  the  variables  and  plan  for  maximum 
choice,  the  true  outcome  of  learning  tailored  to  support  an  intrinsic  motivated 
learner  will  remain  unknown.  In  this  scenario,  elements  of  strategies  from  the 
ARCS  model,  self-regulation,  goal  orientations,  and  the  requirements  of  the 
intrinsic  motivated  learner  are  used  to  maximize  autonomy,  relevance,  and 
competence.  This  all  points  to  the  direction  that  motivation  with  respect  to  learning 
is  connected  to  multiple  variables  and  instructional  strategies.  The  goal  for  the 
present  work  is  to  align  the  variables  (learner  individual  differences)  and  specific 
strategies  an  individual  is  learning  to  achieve  optimum  levels  of  motivation  for  an 
ITS. 

It  is  possible  that  certain  personalities  will  map  to  be  intrinsic  learners.  When  doing 
a  mapping  of  personality  and  thinking  about  each  personality.  The  variables  of  an 
intrinsic  learner  interact  with  the  personality  traits  of  high  conscientiousness,  low 
neuroticism,  and  high  openness.  It  is  possible,  that  high  extraversion  is  related  to  a 
higher  well-being,  happiness,  and  general  self-efficacy  might  also  play  a  role  in 
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being  an  intrinsic  learner.  A  prediction  is  the  following  personalities  might  attribute 
to  higher  intrinsic  tendencies: 

1)  high  openness  +  high  conscientious  +  high  extraversion  +  high  agreeable  + 
low  neuroticism 

2)  high  openness  +  high  conscientiousness  +  high  extraversion  +  low 
agreeableness  +  low  neuroticism 

3)  high  openness  +  high  conscientiousness  +  low  extraversion  +  low  agreeable 
+  low  neuroticism 

4)  high  openness  +  high  conscientiousness  +  low  extraversion  +  high 
agreeableness  +  low  neuroticism 

If  the  MAT  does  correlate  with  personality,  then  possibly  the  personality  test  alone 
could  automatically  place  a  learner  on  an  intrinsic  learner  path.  The  rest  of  the 
personality  mapping  ideas  are  in  the  extrinsic  learner  plan. 

3.2.1  Idea  Development  for  Quadrant  1:  Intrinsic  Learner 

Table  3  shows  an  overview  of  the  intrinsic  learner  plan  for  Quadrant  1  personalities. 
These  are  levels  of  adaptations  that  an  ITS  could  supply  to  this  type  of  learner  based 
off  the  MAT  or  personality. 

Table  3  Intrinsic  learner  support  overview 

Support  Low 

Support  Type  Maximize  autonomy 

Goal  Write  own  or  select  the  High  complexity 

mastery  goal 

Feedback  Correct/ incorrect  feedback  1)  Incorrect  answer  is  shown  the  correct  way. 

This  will  stop  faulty  mental  models.  Learner 
can  choose  to  write  about  their  mistake. 

Task  Pick  the  type  of  task:  1)  Traditional  active  task 

2)  Playground-problem/scenario  based 

3)  Allow  them  to  find  and  create  fif  applicable) 

Reinforcer  Surprise  noncontingent 
reinforcer  at  end 

The  process  flow  is  generic  to  any  ITS.  It  serves  as  an  example  for  a  level  of 
adaptation  made  for  the  intrinsic  learner. 
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The  process  flow  in  Fig.  1 1  describes  a  typical  flow  a  lesson  plan  with  adaptations 
made  for  an  intrinsic  learner.  This  flow  begins  giving  choices  to  the  intrinsic/high 
autonomy  learner.  Motivation  occurs  from  autonomy,  competence,  and  relevance 
for  an  intrinsic  learner.  Basic  adaptive  authoring  will  be  used  for  Year  2  because 
instructors  that  author  courses  are  not  computer  engineers.  They  will  first  need  to 
author  a  basic  course  flow  before  learning  the  aspects  of  implementing  MAT  into 
a  more  stimulated  environment.  A  simulated  environment  through  gaming 
situations  with  MAT  is  perhaps  a  focus  of  Phase  3.  The  adaptation  effects  to 
learning  will  first  be  mastered  within  an  instructor’s  capability  with  technology. 


Fig.  11  Intrinsic  learner  process 

3.2.2  Prior  Knowledge,  Goal  Setting,  Preassessment,  and  Plan 

In  planning  of  self-regulation,  learners  activate  prior  knowledge,  set  goals,  and 
monitor  progress  (Schraw  et  al.  2006).  The  intrinsic  learner  plan  will  provide  the 
learner  with  an  option  to  demonstrate  their  prior  knowledge  before  they  begin  the 
course  by  providing  3  options:  1)  requesting  a  preview/summary  of  information 
needed  prior  to  the  task,  2)  sharing  their  prior  knowledge,  or  3)  opting  out  because 
of  their  high  level  of  knowledge  on  the  topic.  Learners  who  lack  prior  knowledge 
and  have  intrinsic  motivation  will  likely  select  the  first  option  to  review  the  lesson 
content  before  starting  the  lesson.  Learners  also  have  the  option  to  select  the  format 
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of  the  lesson  content  preview — such  as  either  reading  text,  watching  a  video,  or 
listening  to  an  audio  script.  The  second  option,  to  share  their  prior  knowledge,  can 
be  implemented  by  having  the  students  record  a  video/audio  file  or  writing/typing 
a  summary  of  their  knowledge.  The  last  option  monitors  the  learner’s  understanding 
of  the  topic.  If  they  have  a  high  understanding  of  the  topic,  they  will  skip  the  prior 
knowledge  because  of  the  quick  activation.  This  completes  the  first  choice  pathway. 
The  next  section  gives  choices  regarding  the  goal  and  value  of  the  topic. 

Learning  goal  preference  is  a  variable  of  motivation  used  in  the  MAT  to  ensure  the 
intrinsic  motivated  learner’s  goals  align  with  their  preferences.  Goals  increase  the 
commitment  level  of  the  learner  resulting  in  them  extending  more  attention  and 
effort  (Keman  and  Lord  1990).  The  goal  needs  to  be  provided  at  the  correct  level 
of  challenge.  Intrinsic  motivated  learners  typically  have  a  goal  of  complete 
understanding  and  absorption  of  knowledge,  and  may  want  to  write  their  own  goal. 
If  they  are  unsure  on  how  to  write  a  goal,  they  also  have  the  option  to  pick  a  goal 
that  will  meet  their  level  of  entry  and  continue  to  challenge  the  learner. 

Mastery  goal  orientation  is  has  connections  with  the  tendencies  of  an  intrinsic 
learner.  The  mastery  goal  learner  is  after  seeking  knowledge  and  competence.  It  is 
tied  to  positive  affect  for  learning  and  picking  something  more  complex  as  a  task 
(Farr  et  al.  1993).  This  type  of  learner  is  also  related  to  self-efficacy  (Bell  and 
Kozlowski  2002)  and  grit  (Colquitt  and  Simmering  1998)  and  frequently  uses 
metacognitive  strategies  (Ford  et  al.  1998).  Thus,  the  learner  has  an  increased  level 
of  motivation  throughout  the  task  (Colquitt  and  Simmering  1998).  This  goal  is 
chosen  for  the  intrinsic  learner  because  of  the  interconnections  of  the  mastery  goal 
orientations  and  to  the  fit  of  this  type  of  learner.  The  personalities,  mapped  to  this 
quadrant  also  would  be  linked  to  mastery  approach.  Providing  a  learner  autonomy 
the  learner  can  bypass  the  goal  and  write  their  own  goal  to  attain.  This  also  provides 
relevance  to  the  learner. 

Motivation  is  affected  by  forcing  individuals  to  relearn  sections  of  information 
already  established.  If  the  learner  is  knowledgeable  on  the  topic,  then  the 
preassessment  not  only  allows  them  to  skip  sections  but  also  gauges  time 
management.  ITS  will  prompt  the  learner  to  reflect  if  the  preassessment  will  be 
valuable  to  them.  Administration  of  the  preassessment  should  only  transpire  if  they 
are  knowledgeable  on  the  subject.  Presenting  a  plan  according  to  the  data 
accumulated  spells  out  areas  of  weakness  and  strength.  The  learner  then  has  the 
option  to  skip  areas  of  strength.  The  freedom  to  pick  sections  they  feel  confident 
without  a  preassessment  is  not  provided  because  of  the  research  showing  inaccurate 
self-assessment  of  a  learner’s  knowledge  (Brown  2001).  Thus,  a  low-risk 
preassessment  is  used  to  provide  accurate  information  when  deciding  on  an  action 
plan.  This  furthers  the  pathway  for  allowing  intrinsic  motivated  learners  to  have  a 
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voice  and  choice  on  their  plan  for  learning.  In  contrast,  low  knowledge  of  the 
subject  wastes  precious  learning  time  by  taking  the  preassessment. 


3.2.3  Task  and  Practice 

Usually  a  lesson  is  designed  so  that  all  students  learn  within  the  same  type  of 
design — either  a  common  scenario  or  problem  to  solve.  Due  to  the  relevance  of 
deep  learning  and  need  for  student  autonomy,  it  is  recommended  that  the  learner  be 
provided  options  on  how  the  instruction  is  provided  to  the  student.  Traditional 
delivery,  problem  based/scenario  based,  and  “create  my  own  if  applicable”  were 
chosen  because  of  the  level  of  student-centered  learning  those  tasks  provide.  By 
providing  some  variance  in  the  traditional  task,  the  possibility  to  explore 
differences  between  different  types  of  task  motivation,  retention,  and  personality 
selection  is  enabled.  A  traditional  lesson  of  PowerPoint  will  be  used  to  gauge  the 
effect  of  personalization  through  a  traditional  task  of  providing  information  and 
then  practicing.  It  will  still  be  personalized  on  complexity,  preference,  chunking, 
and  so  on. 

The  intrinsic  motivated  learner  should  be  able  to  select  the  level  of  guidance  they 
need  to  begin  practice  based  off  their  competence  of  the  subject.  Consequently, 
recommending  that  the  intrinsic  motivated  learner  be  presented  with  3  options  for 
instructional  guidance.  The  first  level  is  to  provide  no  guidance  to  the  learner  to 
help  them  with  the  task.  The  next  level  of  guidance  provides  hints  to  the  learner.  In 
this  case,  if  the  learner  is  competent  they  can  walk  through  most  of  the  steps  for 
answering  the  questions,  but  may  need  assistance  at  different  points.  The  last  level 
is  the  maximum  level  of  guidance.  The  ITS  walks  the  learner  through  the  entire 
process  and  reasoning  with  the  learner  to  answer  the  question.  These  levels  follow 
the  zones  of  proximal  development  (Vygotsky  1978)  of  the  “I  do,  we  do,  you  do”. 
If  the  learner  provides  3  wrong  answers  in  a  row,  the  ITS  can  automatically  kick 
them  to  the  previous  level  of  guidance.  However,  3  correct  answers  in  a  row  would 
move  them  to  the  less  guidance  on  their  own.  Three  correct  answers  on  own  without 
help  will  move  the  learner  out  of  the  assessment.  The  3  questions  asked  are  of  easy, 
medium,  and  hard  complexity  level  based  off  of  Bloom’s  taxonomy  (Bloom  1984). 
This  speeds  up  the  practice  and  there  is  no  set  amount  of  problems  set  to  pass  the 
practice  section.  The  computer  will  offer  more  time  to  practice  if  the  learner  feels 
they  need  it  and  this  way  they  will  be  self-monitoring  their  progress  (Zimmerman 
2001). 

Practice  is  a  low-risk  formative  assessment  because  there  is  not  a  score  provided 
for  this  time.  Low-risk  assessment  is  needed  to  maintain  motivation,  especially  if 
the  intrinsically  motivated  learner  is  low  on  entry  on  the  topic  and  confidence. 
Intrinsic  motivated  learners  will  only  be  shown  pass/fail  grade  in  GIFT.  Passing 
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practice  would  provide  a  passing  grade  to  the  learner.  This  increases  the  level  of 
success  and  decrease  the  level  of  control  an  assessment  brings.  Data  collection  is 
seen  through  the  practice  session.  Learners  are  able  to  practice  until  they  feel 
competent  and  have  a  deep-level  learning  and  beat  the  different  levels  of 
complexity  independently. 

3.2.4  Reflection,  Competence,  and  Surprise 

Some  learners  naturally  reflect  on  their  own.  Some  are  curious  to  see  how  others 
reflected  and  if  they  missed  any  key  points  in  their  learning.  Others  do  not  like  to 
reflect,  but  would  rather  hear  a  proper  reflection.  Reflection  is  an  important  part  of 
processing  the  information.  This  is  a  prompted  self-regulation  skill.  Learners  could 
reflect  on  themselves  or  their  learning.  The  intrinsic  learner  has  already  achieved 
in  passing  and  reflected  deeply  on  their  learning.  This  competence  section  is  aimed 
at  the  intrinsic  level  of  satisfaction  from  the  amount  of  information  that  was 
provided.  They  may  want  to  continue  learning  or  deepen  their  understanding,  which 
equates  better  retention.  They  can  choose  from  researching  more  on  their  own  time, 
learn  more  about  the  topic,  or  learned  enough  about  the  topic.  The  intelligent  tutor 
will  need  to  know  if  the  learner  has  enough  satisfaction  in  the  amount  of 
information  that  was  provided.  A  noncontingent  reward  during  the  task  or  practice 
will  be  used  to  increase  motivation.  This  might  be  a  motivational  quote,  funny 
animation,  or  a  piece  of  candy.  While  external  motivators  are  needed  by  the  learner 
with  extrinsic  motivation,  the  fact  that  it  is  noncontingent  means  it  will  not 
undermine  intrinsic  motivation  but  will  increase  motivation. 

The  intrinsic  scenario  is  built  to  open  traditional  methods  into  more 
student-centered  practices  by  giving  choice,  voice,  and  relevance  to  their  learning. 
It  also  provides  adjustments  to  meet  their  level  of  need  for  learning.  Intrinsic 
learning  intertwined  with  autonomy,  self-regulation,  motivational  design,  and  the 
universal  design  of  learning  is  used  to  see  the  potential  increase  in  motivation  and 
the  effect  it  has  on  learning  rate  and  retention.  See  future  sections  for  further 
considerations  on  increasing  student  voice,  relevance,  and  instructional  strategies 
to  GIFT. 

The  main  ideas  implemented  in  the  intrinsic  learner  plan  can  be  carried  over  to  use 
with  the  extrinsic  learner  plan.  For  example,  a  video  or  the  main  task  can  be  used 
in  the  extrinsic  learner  plan,  but  have  more  motivators  involved  in  the  adaptations 
of  the  course.  This  way,  authoring  the  2  separate  adaptations  will  not  require  much 
more  time  for  the  instructor.  The  Phase  2  experiment  will  pick  the  most  important 
pieces  of  the  intrinsic  learner  plan  to  be  carried  out. 
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3.3  Quadrants  2-4:  Extrinsic  Learner  Plan 


This  scenario  is  dependent  on  the  level  of  support  their  personality  mapping  falls 
under  or  the  more  detailed  specifics  answered  from  the  MAT.  Learners  will  have 
various  levels  of  specific  feedback,  time  schedule  of  when  reinforcers  are  used,  and 
different  levels  of  support  based  on  their  personality.  It  is  not  so  clear-cut  for 
extrinsic  learners.  Some  researchers  disagree  with  applying  external  rewards  (Deci 
et  al.  2001),  others  agree  that  external  rewards  bring  benefit  the  learner  (Gagne  and 
Deci  2005)  and  are  not  always  harmful  (Eisenberger  et  al.  1999). 

Society  commonly  imposes  nonintrinsic  tasks  on  a  learner.  However,  Ryan  and 
Deci  (2000)  posit  that  humans  are  bom  as  intrinsic  learners.  Habits  are  formed  and 
some  learners  now  expect  extrinsic  reinforcers  in  exchange  for  outcomes  in 
addition  to  the  topic  simply  being  of  interest.  Therefore,  both  expectations 
developed  from  experience  and  interest  in  the  topic  is  understood.  External 
motivation  shifts  the  focus  to  outside  significance  for  learning  in  combination  with 
an  internal  gratification,  such  as  success.  Acknowledgement,  authority,  peer 
pressure,  grades,  and  the  promise  of  a  reward  becomes  the  focal  point  of  external 
factors.  A  task  that  is  extrinsically  motivating  produces  a  different  entry  level  of 
motivation.  This  motivation  is  harder  to  maintain  because  success  relies  upon 
external  factors  for  completion  that  matches  the  individual’s  level  of  expectation 
and  preference.  An  extrinsic  learner  is  attempting  to  gain  or  avoid.  Extrinsic 
motivation  that  still  provides  the  learner  a  level  of  control  is  able  to  provide  a 
learner  with  higher  levels  of  performance  (Miserandino  1996)  and  well-being 
(Sheldon  and  Kasser  1999).  Extrinsic  learners  are  either  strategic  learners  or  surface 
learners  (Cassidy  and  Eachus  2000).  A  strategic  learner’s  intention  is  for  the  highest 
grade  possible  and  a  surface  learner  is  to  learn  the  minimum  requirements. 

Extrinsic  motivation  can  be  broken  down  into  a  continuum  known  as  Organismic 
Integration  Theory  (OIT;  Ryan  and  Deci  2000,  Fig.  12).  Starting  with  internal 
control  known  as  integration  (regulation  has  been  integrated  within  their  sense  of 
self),  identification  (person  sees  the  importance  of  the  activity),  introjection 
(controlled  by  inside  factors  such  as  self-esteem),  and  external  regulation 
(controlled  by  demands  or  contingencies;  Ryan  and  Deci  2000).  Integrated  falls 
very  high  on  self-regulation  (Deci  et  al.  1996).  Identified  falls  as  being  part  of  an 
intermediary  level  of  self-regulation  because  of  the  importance  of  the  activity. 
Introjected  has  a  moderately  low  level  of  self-regulation  because  of  being 
controlled  inside  (e.g.,  self-esteem)  and  external  is  very  low  and  needs  to  be 
controlled  by  demands.  The  MAT  might  see  similar  patterns  from  individuals  or 
provided  that  self-regulation  is  a  finer  tailoring  to  the  individual  and  individuals 
may  have  a  compilation  of  those  due  to  the  personality  mapping. 
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Fig.  12  Continuum  of  an  extrinsic  learner 

The  adaptive  course  flow  is  complicated  for  the  external  learner  in  comparison  to 
an  intrinsic  learner.  The  level  of  complexity  is  increased  due  to  the  need  to 
continuously  maintain  motivation  to  enable  a  higher  level  of  academic  performance 
and  retention.  The  requirements  of  an  extrinsic  learner  layered  with  other 
motivation  interconnections  and  personality  is  dynamic  with  multitudes  of 
motivators  to  select.  For  Phase  2,  a  select  set  of  motivators  will  be  validated, 
possibilities  are  tokens,  timers,  acknowledgement,  leaderboards,  or  a  tangible  item. 
The  frequency,  control,  and  support  level  are  drawn  out  within  the  different 
quadrants  (pending  on  MAT  results).  If  there  is  no  connection  to  personality,  then 
the  plan  will  be  dependent  upon  individual  rules  as  seen  in  the  individual  rule 
section  from  the  MAT. 

A  task,  extrinsic  to  the  learner,  develops  variance  of  value  to  learning  the  task.  This 
level  of  value  does  not  always  match  a  motivational  level  for  maximizing  deep 
learning  engagement.  Value  for  some  extrinsic  learners  resides  in  the  outcome 
provided  from  achieving  the  outcome  and  not  the  actual  learning  that  occurs.  Thus, 
options  for  some  quadrants  are  removed.  Since  learners  might  not  have  the 
motivation  to  be  deep  learners,  they  need  to  have  more  review  and  more 
accountability  for  the  information.  If  they  do  not  meet  their  goal  by  the  end  of  the 
assessment,  the  course  is  not  over  and  the  computer  will  automatically  flip  the 
learner  into  a  different  task.  Maybe  the  task  was  not  the  correct  fit  for  the  learner  to 
process  the  information.  The  learner  does  not  need  to  be  aware  of  having  to  do  the 
same  process,  but  it  could  feel  like  the  learning  task  continued  after  the  questions. 
This  may  increase  the  learning  rate,  but  it  is  important  that  the  extrinsic  learner 
reach  success  at  the  end  of  the  course. 
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3.3.1  Idea  Development  for  Quadrant  2:  High  Neuroticism 

The  second  quadrant  are  learners  high  in  neuroticism,  but  also  high  in  conscientious 
and  openness  (Fig.  13).  On  the  MAT,  these  individuals  are  expected  to  produce 
results  consistent  with  struggling  with  their  emotional  well-being  and  anxiety,  but 
with  a  high  level  of  effort  and  interest.  Emotional  instability  is  a  contributor  to  low 
academic  achievement  and  avoidance.  A  need  for  emotional  support  and 
competence  become  evident  when  mapping  this  type  of  person’s  other  traits.  This 
falls  in  line  with  introjected  self-regulation  where  one  deals  with  issues  such  as  self¬ 
esteem.  The  level  of  self-regulation  is  moderately  low  due  to  the  emotions  being 
kept  inside  (Deci  et  al.  1996).  It  also  falls  in  line  with  integrated  and  identified 
regulation  because  of  their  openness  and  conscientiousness.  They  already  come 
valuing  the  task  and  achievement.  The  learner  may  have  a  difficult  time  with  their 
emotions  and  giving  up  (avoidance),  but  is  balanced  by  a  drive  for  perfection  and 
achievement,  and  open  to  experiences  and  learning  new  ideas.  This  type  of  person 
battles  to  overcome  their  emotions  and  constant  need  to  feel  successful.  They  have 
the  tendency  to  be  mastery  avoidant  or  performance  avoidant,  but  link  to  a  mastery 
approach  in  conscientiousness  (Elliot  and  Thrash  2002).  Steering  away  from 
avoidance  behavior  is  key.  Added  support  is  suggested  to  tend  to  their  specific 
requirements  for  competence,  emotional  support,  and  success.  Extraversion  and 
agreeableness  is  shown  in  groups  to  depict  an  even  more  granular  refinement  of 
motivating  the  individual.  Both  types  of  high  and  low  extraversion  are  capable  of 
high  academic  achievement.  They  are  split  into  groups  by  those  factors  that 
determine  competitive  and  learning  preference  (individual  vs.  team)  as  seen  and 
explained  further  in  Fig.  13. 

High  agreeable  and  low  extraversion 

high  openness  +  high  conscientiousness  +  low  extraversion+  high  agreeable  +  high  neuroticism 
high  openness  +  high  conscientiousness  +  low  extraversion  +  high  agreeable  +  high  neuroticism 

High  agreeable  high  extraversion 

high  openness  +  high  conscientiousness  +  high  extraversion+  high  agreeable  +  high  neuroticism 
high  openness  +  high  conscientiousness  +  high  extraversion  +  high  agreeableness  +  high  neuroticism 

'  Low  agreeableness  and  low  extraversion 

high  openness  +  high  conscientiousness  +  low  extraversion+  low  agreeable  +  high  neuroticism 
High  openness  +  high  conscientiousness  +  low  extraversion  +  low  agreeableness  +  high  neuroticism 

'  Low  agreeableness  and  high  extraversion 

high  openness  +  high  conscientiousness  +  high  extraversion+  low  agreeable  +  high  neuroticism 
high  openness  +  high  conscientiousness  +  high  extraversion  +  low'  agreeableness  +  high  neuroticism 


Fig.  13  Hypothesized  personality  mapping  for  Quadrant  2 
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This  learner  needs  to  remain  competent  and  express  their  feelings  intertwined  with 
the  task.  They  generally  have  lower  self-esteem,  self-efficacy,  perhaps 
strategic/surface-level  learning  because  of  the  emotions.  However,  they  are  battling 
against  persistence,  grit,  and  creativity.  Driven  but  vulnerable,  their  plan  needs  to 
be  cautious  and  emotionally  positive  based.  The  task  needs  to  build  from  a  low 
level  to  a  more  complex  level  through  subgoals  and  always  remain  on  a  level  of 
success.  Mistakes  are  worrisome  and  need  to  have  the  mindset  of  feedback  for 
growth  and  not  a  punitive  measure.  This  learner  might  retain  better  with  expressing 
their  feelings  and  intertwining  it  into  the  lesson.  It  will  be  important  to  set  up  the 
reflection  in  this  direction. 

Table  4  Quadrant  2  support  overview 


Support 

Level 


Medium 


Support  Emotional  and  competence 

Type 

Goal  Subgoals 


Feedback 


Correct/incorrect  feedback 

Empathetic 

Growth  mindset 


Task 


Pick  the  task: 


Motivator 


Surprise  noncontingent 
reinforcer  relaxation  stress 
reducing 


Introjected  regulation  because  of  self-esteem 
and  avoidance.  However,  they  are  also 
integrated  and  identified  regulation  because 
of  value  and  needs  seen  by  the  learner. 

Low  complexity 


1)  Incorrect  answer  will  be  shown  the 
correct  way.  This  will  stop  faulty  mental 
models. 

2)  Learner  can  choose  to  write  about  their 
mistake. 

3)  Empathetic  and/or  growth  mindset  can 
be  added  to  the  comment  or  as  a  separate 
entity. 

1)  Traditional  active  task 

2)  Playground-  problem/scenario  based 

3)  Game 


Build  competence  at  beginning, 
(e.g..  You  will  be  successful  if 
you  put  the  effort  forth!) 


Breaks  to  encourage  relaxation 

More  reviews  and  reflection 
with  an  emotional  attachment 


Motivators  -  less  and  bigger 
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The  process  flow  in  Fig.  14  needs  to  include  relaxation  techniques  and  building 
competence  (positive  mindset  and  self-talk),  and  remain  at  the  lower  level  of  this 
learner’s  knowledge  to  build  success  (Table  5).  The  learner  should  not  take  the 
preassessment  after  3  answers  wrong  because  getting  more  mistakes  might  make 
them  avoidant  or  frustrated.  They  can  pick  the  type  of  task  that  is  broken  into 
subskills.  Feedback  that  is  empathetic  and  of  a  growth  mindset  is  planned  alongside 
correct  and  incorrect  answers.  Incorrect  answers  must  not  be  called  a  mistake,  but 
learning.  The  practice  should  always  begin  with  language  such  as  I  do  (learner 
watch),  we  do  (learner  has  hints  available  to  help),  you  do  as  a  novice  level,  and 
brought  up.  They  do  not  need  to  know  that  they  are  starting  at  a  novice  level.  Since 
they  are  high  in  conscientiousness  and  openness,  the  review  needs  to  tie  in  emotions 
for  remembering  or  competence.  This  can  be  set  into  the  guiding  questions  for 
review  on  having  them  reflect.  Then  provide  a  relaxation  technique,  breathing,  or 
mindfulness  to  help  reduce  their  anxiety  and  build  up  their  competence  on  the  task 
with  a  quote  or  short  video.  To  see  their  depth  of  knowledge  versus  a  traditional 
task,  learners  will  write  short  responses  or  extended  responses  that  are  graded  on  a 
rubric.  Learners  in  this  quadrant  will  be  guided  to  review  main  points  learned  within 
a  relaxation  exercise  reflect  on  experience  and  successes  of  what  they 
comprehended.  If  a  learner  doesn’t  understand  the  information  they  can  be  sent  to 
another  task.  This  can  be  done  on  each  subskill  as  the  formative  assessments  or 
recall  determine  readiness  to  reach  another  subskill  toward  the  goal. 


Relaxation 
Build  Competence 

How  to  earn  reinforcers? 
Reassure  they  will  be  sucessful 


Prior  Knowledge  (no  choice) 

A)  video 

B)Read  what  others  know 
C)  What  do  you  know? 


Goal: 

Mini  sub-goals 


l.Traditional  Task-  active 

2.  Problem  Scenario  based  task 
-playground 

Pick  different  scenarios  developed  to  learn 
different  parts 

Practice  puts  them  together  as  steps  of  a 
whole. 

3.  Game 


Practice-  Placed 

Review  main 

throughout  subparts  of 

points  and  details 

task. 

from  this  topic/ 

They  will  always  enter 

reflect  on  success 

^novice  at  the  I  do  level. 

of  what  they 

Levels  Novice, 

learned  thus  far 

Journeyman,  expert 

throughout  task. 

3  correct  expert  move 

up,  3  wrong  move 

Relaxation 

down  a  level 

technique 

Review  main 
points  and 
details  from 
this  topic 
-  Relax 


Reflection 

Learner  Shares  an  emotional  tied 
reflection  on  what  they  learned. 

End  of  reinforcers  and  score  pass  then 
Finished.  Didn’t  meet  the  goal?  Learner 
retakes  a  different  task. 


Assessment 
Depth  of 
Knowledge 
Assessment  with  an 
emotional  tie 


Fig.  14  Quadrant  2  process 
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3.3.2  Idea  Development  for  Quadrant  3:  Low  Conscientiousness  or 
Openness 

The  third  quadrant  is  broken  into  learners  that  are  low  in  conscientiousness  or 
openness.  These  learners  need  more  support  in  focusing,  organization,  patience, 
grit,  and  steering  away  from  procrastination.  Low  openness  yield  traits  such  as  less 
abstract  thinking,  low  on  intrinsic  motivation,  less  willing  to  change,  and  high  on 
sensitivity  to  punishment  because  of  sensitivity  to  cues  of  novelty.  Focusing  on 
strategies  that  tailor  to  less  abstract  thinking,  low  motivation,  and  maintaining 
tradition.  These  personalities  were  grouped  with  learners  that  are  in  combination 
with  low  neuroticism. 

The  mapping  is  as  follows: 

Quadrant  3:  One  area  of  low  in  either  conscientiousness  or  openness  personalities: 
High  agreeable  and  low  extraversion 

1)  Low  openness  +  high  conscientiousness  +  low  extraversion  +  high 
agreeable  +  low  neuroticism 

2)  High  openness  +  low  conscientiousness  +  low  extraversion  +  high  agreeable 
+  low  neuroticism 

High  agreeable  high  extraversion 

3)  Low  openness  +  high  conscientiousness  +  high  extraversion+  high 
agreeable  +  low  neuroticism 

4)  High  openness  +  low  conscientiousness  +  high  extraversion  +  high 
agreeableness  +  low  neuroticism 

Low  agreeableness  and  low  extraversion 

5)  Low  openness  +  high  conscientiousness  +  low  extraversion+  low  agreeable 
+  low  neuroticism 

6)  High  openness  +  low  conscientiousness  +  low  extraversion  +  low 
agreeableness  +  low  neuroticism 

Low  agreeableness  and  high  extraversion 

7)  Low  openness  +  high  conscientiousness  +  high  extraversion+  low  agreeable 
+  low  neuroticism 

8)  High  openness  +  low  conscientiousness  +  high  extraversion  +  low 
agreeableness  +  low  neuroticism 
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Table  5  shows  the  overview  for  Quadrant  3.  While  this  learner  is  emotionally  stable, 
they  are  low  in  conscientiousness  or  openness.  They  are  not  in  a  battle  over  their 
emotions  but  with  relevance,  this  task  holds  to  them.  They  need  support  in  the  area 
of  creativity,  organization,  abstract  concepts,  focus,  and  reviewing  more  than  once 
because  of  lack  of  interest  or  detail.  This  person  needs  a  higher  level  of  support 
than  above  because  of  their  lack  of  interest,  procrastination,  organization,  effort  to 
avoid  abstract  ideas,  and  thinking  outside  the  box. 

Table  5  Quadrant  3  support  overview 

Support  Level  Medium  to  medium/  high 

Support  Type  Organize,  detail,  push  to  abstract, 
repetitive  review 

Goal  Subgoal,  value  of  task  Begins  low  complexity 

Feedback  Correct/  incorrect  feedback  1)  Incorrect  answer  will  be  shown 

the  correct  way.  This  will  stop 
Focus,  self-regulation  for  organization  faulty  mental  models.  Learner 

can  choose  to  write  about  their 
mistake. 

Task  Pick  the  type  of  task:  1)  Traditional  active  task 

2)  Playground-problem/scenario 
based 

3)  Game 

Low  focus  parts  of  different  task  that 
yield  higher  stimulation 

More  review  sections 

Forced  push  to  abstract 

Forced  reflection 

Reinforcer  Motivation  begins  at  start  of  task  (e.g., 
noncontingent) 

More  frequent  smaller  amount 
Punitive:  loss  of  points  (extremely  too 
long,  way  too  many  wrong) 

Figure  15  provides  their  overall  process  flow  for  a  Quadrant  3  learner  that  is  low 
conscientiousness  or  openness.  Motivation  needs  to  be  boosted  from  the  beginning. 
This  learner’s  noncontingent  reward  is  given  at  the  beginning  of  the  task.  Value  of 
the  task  must  be  provided  because  this  learner  does  not  see  the  relevance  of  this 
task.  Clear  rules  and  expectations  for  the  rein  forcer  being  used  is  also  given.  This 
learner  will  watch  a  prior  knowledge  video  and  will  have  to  write  what  they  learned. 

Approved  for  public  release;  distribution  is  unlimited. 

41 


They  are  not  emotional  learners  so  they  will  take  the  preassessment  to  test  their 
knowledge  and  pass  out  of  sections.  Their  goal  will  be  in  subgoals  and  meeting 
progress.  The  topics  will  be  broken  into  subgroups  each  subgroup  will  contain  the 
practice  or  formative  assessment.  The  learner  can  pick  the  task.  This  includes  a 
game  since  games  have  extrinsic  features.  If  the  learner  does  not  pass  the  first 
subtask  they  can  take  the  subtask  of  a  different  task  and  repeat  until  they  meet  the 
achievement  of  the  subgoal  for  that  section.  A  review  will  be  given  to  remind  the 
learner  of  the  important  points  and  details  learned  in  that  subtopic.  A  small  break 
to  refocus  their  attention  will  be  given  through  physical  movement  or  animation 
video  about  working  hard.  Organizers,  classifiers,  and  focus  reflection  will  be  given 
to  help  deepen  the  understanding  of  the  learner.  The  learner  will  proceed  through 
the  subtask  until  completion.  Completion  occurs  after  they  meet  their  subgoal  and 
assessment.  A  final  review  and  reflection  of  all  the  things  learned  is  planned. 


Fig.  15  Quadrant  3  process 

3.3.3  Idea  Development  for  Quadrant  4:  Low  Conscientiousness  and/or 
Openness  and/or  High  Neuroticism 

Quadrant  4  is  the  learners  who  need  the  most  support  because  they  have  multiple 
lows  in  either  conscientiousness  and/or  openness  and/or  high  neuroticism.  This 
quadrant  is  predicted  to  be  the  learners  that  need  external  control  and  continuous 
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support.  They  are  expected  to  represent  the  external  regulation  or  amotivation 
sections  of  the  SDT  (Ryan  and  Deci  2000).  They  need  both  emotional  support  and 
organizational,  focus  support.  While  there  are  more  of  these  personality  maps,  it  is 
predicted  that  this  also  makes  up  the  smallest  percent  of  the  population.  They  might 
have  become  removed  from  an  intrinsic  learner  because  of  past  experiences  that 
initiated  low  self-efficacy,  low  self-esteem,  and  carelessness.  It  has  been  noted  that 
some  of  these  personality  trait  strings  make  up  disabilities  such  as  ADHD.  Learners 
struggling  to  focus  or  holding  still  are  attributed  to  the  personality  traits  being  high 
in  neuroticism,  high  in  extraversion,  low  in  agreeableness,  and  low  in 
conscientiousness  (Mitchell  and  Nelson-Gray  2006).  These  learners  need 
motivation  from  the  beginning.  They  need  a  task  that  gives  them  success  at  a  less 
complex  level  to  start  building  their  self-esteem  and  self-efficacy.  They  need  to 
learn  the  material  in  multiple  ways  that  incorporate  repetition.  They  need  to  have 
more  guided  direct  instruction  before  receiving  practice.  Knowledge  that  this 
person  has  low  tolerance  and  can  shut  down  over  just  a  simple  mistake.  They  need 
to  stress  accomplishments  and  visuals.  This  learner  might  fall  into  the  “at  risk”  tier. 
The  4  underlined  are  the  most  extreme  out  of  the  set  of  personality  mappings: 

High  extraversion  low  agreeableness 


1)  high  openness  +  low  conscientiousness  +  high  extraversion  +  low 
agreeableness  +  high  neuroticism 


3)  low  openness  +  high  conscientiousness  +  high  extraversion  +  low 
agreeableness  +  high  neuroticism 

4)  low  openness  +  low  conscientiousness  +  high  extraversion  +  low 
agreeableness  +  low  neuroticism 

High  extraversion  high  agreeableness 


6)  low  openness  +  low  conscientiousness  +  high  extraversion  +  high 

agreeableness  +  low  neuroticism 

7)  low  openness  +  high  conscientiousness  +  high  extraversion  +  high 

agreeableness  +  high  neuroticism 

8)  high  openness  +  low  conscientiousness  +  high  extraversion  +  high 

agreeableness  +  high  neuroticism 
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Low  extraversion  and  low  agreeableness 


9)  low  openness  +  high  conscientiousness  +  low  extraversion  +  low 

agreeableness  +  high  neuroticism 

10)  low  openness  +  low  conscientiousness  +  low  extraversion  +  low 

agreeableness  +  high  neuroticism 

11)  high  openness  +  low  conscientiousness  +  low  extraversion  +  low 

agreeableness  +  high  neuroticism 

12)  low  openness  +  high  conscientiousness  +  low  extraversion  +  low 

agreeableness  +  high  neuroticism 

Low  extraversion  and  high  agreeableness 

13)  low  openness  +  low  conscientiousness  +  low  extraversion  +  high 
agreeableness  +  high  neuroticism 

14)  low  openness  +  high  conscientiousness  +  low  extraversion  +  high 
agreeableness  +  high  neuroticism 

15)  high  openness  +  low  conscientiousness  +  low  extraversion  +  high 
agreeableness  +  high  neuroticism 

16)  low  openness  +  low  conscientiousness  +  low  extraversion  +  high 
agreeableness  +  low  neuroticism 

Quadrant  4  is  the  most  intense,  but  predicted  to  have  the  least  number  of  learners. 
This  is  an  at-risk  learner.  They  are  at  risk  because  they  have  more  than  one  area 
that  is  linked  to  low  academic  achievement.  They  come  in  with  the  lowest  form  of 
regulation  and  motivation  and  are  probably  at  an  external  motivation  category  and 
maybe  amotivated.  They  are  low  in  openness  and/or  conscientiousness  and/or  high 
neuroticism.  They  have  a  difficult  time  with  their  emotions,  focus,  organization, 
competence,  and  vulnerability,  and  do  not  hold  value  for  achievement.  They  need 
maximum  support  of  external  factors  to  help  navigate  through  learning.  Length  of 
learning,  time  spent  on  a  single  task,  stimulation,  color  coding,  depth  of  the 
material,  feedback,  and  motivator  all  need  to  be  considered  when  presenting 
materials.  Table  6  shows  the  factors  from  the  MAT  anticipated  to  be  associated 
with  this  type  of  personality  mapping. 
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Table  6  Quadrant  4  support  overview 


Support  Level 


High 


Support  Type 


Emotional,  focus,  organization  and 
competence 


Low  complexity 


Goal 


Subgoals 


Feedback  Correct/  incorrect  feedback 
Empathetic 
Growth  mindset 
Focus,  self-regulation,  progress 

Task  Mix  up  the  task  based  on  high 

stimulation  of  task: 


1)  Incorrect  answer  will  be  shown 
the  correct  way.  This  will  stop 
faulty  mental  models.  Learner  can 
choose  to  write  about  their 
mistake.  Also,  empathetic  and/or 
growth  mindset  can  be  added  to 
the  comment  or  as  a  separate 
entity. 

1)  Traditional  active  task  use  sub  1 
first  10  min. 

2)  Playground:  problem/scenario 
based  use  sub  1.1  15  min. 

3)  Game  use  sub  1.2  15  min. 

4)  formative  pass  subtask  2  fail 
different  sub  section  1 


Reinforcer 


Surprise  noncontingent  reinforcer 
relaxation  stress  reducing 

Build  competence  at  beginning  (e.g. 
You  will  be  successful  if  you  put  the 
effort  forth!  Value  your  effort  is 
important  because. . .). 

More  frequent  breaks:  video,  relaxation, 
physical  movement 

More  reviews  and  reflection  with  an 
emotional  attachment 


No  preassessment:  we  have  a  limited 
attention  span  and  emotion  (this  may 
waste  important  motivational  effort) 

Begin  at  low  complexity 

More  comprehension  checks 

Organization  by  color 


The  flow  for  this  learner  (Fig.  16)  needs  to  remain  at  a  high  expectation  level  while 
at  the  same  time  providing  the  right  amount  of  information.  Too  much  information 
or  too  deep  will  automatically  lead  to  trying  lack  of  effort.  The  feedback  needs  to 
remain  consistent,  balancing  positive  to  negative  motivators.  Learning  needs  to 
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occur  with  time  in  mind  given  this  learner’s  low  and  limited  motivation.  Therefore, 
the  preassessment  was  removed  because  this  learner  needs  low  complexity  and  to 
remain  in  the  priority  task  at  all  times.  Prior  knowledge  on  the  task  is  limited  to 
5  min.  The  goal  is  a  mini  goal  that  is  shown  throughout  the  task.  This  learner  is 
going  to  need  the  highest  stimulation  of  each  of  the  tasks.  This  is  to  keep  their 
attention  and  motivation.  Breaks  to  relax  and  reset  their  focus  will  likely  occur  more 
frequently.  They  need  the  material  to  be  reviewed  often  to  ensure  learning  for 
comprehension  of  the  material. 


Fig.  16  Quadrant  4  process 

The  ideas  mentioned  previously  are  dependent  on  the  results  from  the  MAT 
validation  waves.  If  strong  relations  occur  between  personality  and  motivation,  then 
these  personality  mappings  can  be  used  to  individualize.  However,  if  strong 
relations  are  not  found,  the  MAT  assessment  will  become  a  rule-based  assessment 
for  providing  specific  information  to  the  learner. 

4.  Recommendations  for  GIFT  Advancements 


GIFT  is  an  adaptive  training  tool  available  free  for  researchers  and,  in  the  future, 
instructors.  Creating  basic  courses  is  easy  to  do  without  having  a  computer 
programming  background.  The  adaptive  courses  are  more  difficult  to  author  due  to 
some  areas  requiring  software  code  or  scripts  to  be  modified  or  written.  Also,  an 
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updated  manual  written  for  new  users  on  all  the  basic  functions  and  more  complex 
functions  within  GIFT  is  needed  as  the  recent  GIFT  updates  are  not  yet  captured  in 
these  documents.  This  would  help  researchers  and  instructors  learn  to  execute 
complex  authoring  for  adaptive  courses.  It  is  also  recommended  that  this  manual 
contain  all  the  information  in  a  single  document  with  hyperlinks  rather  than  the 
current  form  in  which  it  is  spread  out  over  multiple  papers  or  in  disparate  sections 
on  a  Wiki.  An  exemplar  course  developed  in  GIFT  employing  the  different  stages 
of  knowledge/ability  with  GIFT  will  also  help  the  new  GIFT  user.  This  would  save 
ARL  time  and  money  in  addressing  individual  users’  needs  to  learn  GIFT. 
Providing  all  ARL  GIFT  project  teams  with  a  detailed  user  manual  and  upfront 
course,  such  as  the  GIFT  Summer  Camp,  would  be  beneficial  to  learning  the  details 
for  authoring  adaptive  courses. 

This  effort  has  created  some  forum  topics,  such  as  saving  a  long  survey  into  GIFT 
(cloud  version).  The  first  attempt  to  save  the  MAT  resulted  in  an  error  that 
prevented  the  MAT  course  to  be  saved,  so  a  forum  topic  was  created.  While  the 
GIFT  support  team  developed  a  solution  to  fix  the  error,  our  team  employed  a 
workaround  in  which  the  survey  was  broken  into  several  smaller  surveys  so  that 
GIFT  could  support  it.  The  same  error  was  replicated  in  the  downloadable  version 
of  GIFT.  The  GIFT  support  team  has  since  provided  an  update  to  GIFT  (cloud  and 
download  versions)  that  has  corrected  this  issue. 

Another  help  ticket  was  created  by  this  project  regarding  the  output  of  the  data  from 
the  surveys.  When  running  pilot  studies  for  the  MAT  waves,  it  was  discovered  that 
the  computer-generated  tags  associated  with  each  survey  item  response  were  not 
based  on  the  order  of  the  questions  entered  or  the  number  of  the  individual  items. 
Consequently,  it  was  difficult  to  read  the  data  in  Excel  or  other  statistical  programs 
for  analysis.  The  ARL  GIFT  support  team  has  addressed  this  issue.  However,  as  a 
workaround,  our  team  developed  a  batch-processing  script  to  rename  the  item 
response  columns  by  the  survey  tags  and  order  the  columns  by  the  presentation  in 
the  survey.  The  ARL  fix  has  also  recently  been  deployed  to  GIFT  cloud. 

GIFT  Cloud  strives  for  quick  authoring  of  training,  but  our  team  encountered  long 
wait  times  when  saving  or  adding  new  items.  For  example,  opening  up  a  survey 
took  a  minute  to  a  couple  of  minutes.  The  delay  between  each  typing  and  it  showing 
on  GIFT  assessments  took  at  least  30  s.  As  a  workaround  to  improve  efficiency,  the 
questions  from  the  survey  were  typed  in  Qualtrics  and  then  transferred  into  GIFT. 

A  few  authoring  changes  are  recommended.  The  first  recommendation  is  to  provide 
an  option  to  apply  the  same  format,  such  as  a  radio  button  answer  selection,  to  the 
entire  survey,  rather  than  having  to  select  the  answer  selection  for  each  survey  item. 
Another  recommendation  is  to  allow  users  to  save  their  assessments  and  transfer 
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them  to  a  Word  document  for  easier  editing  and  use  in  reports.  In  other  words,  it 
does  not  currently  appear  that  assessments  entered  into  GIFT  can  be  retrieved  in 
any  other  complete  format.  GIFT  allows  users  to  save  their  assessment  within 
GIFT,  but  it  does  not  allow  for  export  outside  of  GIFT. 

GIFT’S  instructional  feedback  options  run  on  a  continuum  of  positive  to  negative. 
However,  it  does  not  appear  that  that  there  is  a  bank  of  different  types  of  feedback 
available  for  tailoring  instruction  to  the  user,  which  would  save  time  in  authoring 
new  courses  and  inform  authors  of  additional  options  that  they  might  otherwise  not 
know.  Different  types  of  feedback  might  include  instructional  feedback,  which 
provides  the  learner  with  information  pertaining  to  their  learning,  and  motivation 
feedback,  which  helps  to  encourage  the  student  in  the  learning  process. 
Instructional  feedback  is  specific  to  the  immediate  learning  task  and  the  learner’s 
performance.  For  example,  instructional  feedback  can  provide  information  to  the 
learner  to  reinforce/explain  correct  performance,  as  well  as  provide  an  explanation 
of  incorrect  performance.  Motivation  feedback  is  used  to  support  the  learner’s 
affect,  self-regulation,  and  effort.  Feedback  is  powerful;  it  fell  in  the  top  5  to  10 
highest  influences  on  achievement  (Hattie  1999)  along  with  direct  instruction  and 
reciprocal  teaching.  According  to  Hattie  (1999),  feedback  works  at  4  levels:  1)  task 
level  (e.g.,  correct/incorrect),  2)  process  level  (e.g.,  remember  to  use  the  strategy 
taught  in  the  course),  3)  self-regulation  (e.g.,  go  back  and  check  to  see  if  you 
completed  the  3  key  steps  in  order),  and  4)  self  (e.g.,  that  was  a  well-thought-out 
response,  well  done).  All  learners  need  feedback  that  at  a  minimum,  states  they  are 
correct  or  incorrect.  Learners  need  to  be  guided  toward  figuring  out  their  mistake. 
To  save  authors’  time,  it  may  be  helpful  to  provide  sentence  starters  or  a  list  of 
different  types  of  feedback  for  the  task. 

It  is  recommended  that  GIFT  make  available  more  extensive  stored  content  for 
reuse.  If  there  was  a  pool  of  materials  to  pick  and  begin,  it  would  drastically 
eliminate  authoring  time.  If  a  free  platform  for  content  was  created  alongside  of  the 
authoring  tool,  it  would  increase  usage  and  drive  it  into  different  contexts  and 
usage.  To  organize  stored  content,  it  is  suggested  to  do  so  by  domain,  curriculum, 
and  standards.  Instructors  pay  a  fortune  on  premade  lessons  to  adapt. 

GIFT  is  a  learner-centered  approach  to  learning  with  games,  simulations,  and 
adaptations  based  on  the  individual.  Other  methods  can  be  applied  for  learner- 
center  approaches.  Assessments  could  be  in  the  form  of  a  presentation  that  is 
recorded  and  then  used  for  reciprocal  teaching  and  free  content  use  by  other 
authors.  This  could  then  be  archived  based  on  level  of  knowledge  the  learner  needs, 
humor,  creativity,  type  of  strategy  used,  and  so  on.  For  example,  social  media 
elements,  such  as  commenting  on  the  video  or  liking  their  video,  may  be  a 
motivator  for  some  students.  Other  instructional  methods  such  as  debate, 
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discussion,  small  group  working  sessions,  jigsaw  learning,  and  workshop  style 
presentation  opportunities  are  all  learner-centered  strategies  that  would  create  deep 
learning  and  might  aid  in  retention.  This  would  enable  the  MAT  to  evolve  beyond 
assessments  with  associated  strategies.  This  would  also  enable  a  flipped  learning 
approach.  Another  area  of  extension  is  to  provide  the  students  an  opportunity  to 
create  their  own  courses  and  have  them  reciprocally  teach,  as  appropriate. 
Reciprocal  teaching  is  known  for  comprehension  of  material  and  a  deeper  level  of 
learning  (Palincsar  and  Brown  1984).  This  could  be  supported  by  having  the 
instructor  available  to  discuss  any  questions  the  learner  might  have  from  their 
course  when  team  teaching  is  added  into  GIFT.  GIFT  is  a  perfect  authoring  tool  for 
this  kind  of  learning  because  of  the  ease  of  use. 
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A-l  Introduction 


The  Army  is  developing  a  framework  for  providing  personalized,  on-demand, 
computer-based  instruction  under  the  Generalized  Intelligent  Framework  for 
Tutoring  (GIFT)  program.  Phase  1  will  focus  on  the  development  of  a  Motivator 
Assessment  Tool  (MAT)  to  enhance  GIFT  that  adapts  its  motivational  strategies 
based  on  individual  differences.  The  MAT’s  assessment  capability  will  identify 
motivation  in  the  learner  in  terms  of  current  level  of  motivation  and  types  of 
motivational  reinforcers  that  work  best  with  the  learner.  It  will  build  upon  efforts 
to  incorporate  additional  classification  variables  that  include  student  personality, 
learning  performance  history,  and  motivational  responses.  Motivational  responses 
refer  to  the  increase  of  sustained  effort,  with  the  end  goal  resulting  in  a  reward  based 
on  personality.  Previous  research  has  shown  that  personality  factors  are  related  to 
motivation.  Similarly,  motivation  has  been  linked  to  the  learning  process.  However, 
the  intersection  between  personality,  the  motivator,  learning  rate,  and  retention  has 
not  been  explored.  A  trainee  who  is  shown  a  reinforcer  based  on  their  personality 
will  enter  a  motivated  state.  When  in  this  motivated  state,  they  will  learn  faster 
because  of  their  deeper  level  of  commitment  and  engagement,  and  also  retain 
information  longer.  While  using  GIFT,  the  reinforcer  given  to  a  person  who  is  high 
in  narcissism  needs  more  frequent  reinforcement  than  a  trainee  who  has  stable 
persistence  during  the  learning  task.  The  focus  of  this  effort  is  to  examine  the 
literature  relevant  to  the  determination  of  the  role  of  individual  differences  in 
motivation  to  learn,  and  the  subsequent  effect  on  motivation  to  learn  on  long-term 
retention. 

A-2  Definition  of  the  Problem 

Motivated  and  engaged  trainees  are  often  in  an  active  state  of  learning  (Isaksen  and 
Hole  2015).  In  this  state,  they  produce  a  higher  sustained  effort  to  reach  a  goal, 
which  enables  them  to  reach  higher  learning  goals  (Ackerman  et  al.  1995;  Knafer 
and  Heggestad  1999;  McCombs  and  Whisler  1989).  In  traditional  classroom 
environments,  the  human  teacher  is  able  to  directly  interact  with  the  student  and 
test  various  strategies  for  motivating  the  student.  However,  in  an  instructorless,  or 
instructor-reduced,  learning  environment  such  as  that  supported  by  GIFT, 
determining  the  constructs  that  motivate  each  individual  is  quite  challenging 
(Blickensderfer  et  al.  2003).  This  is  because  a  human  instructor  receives  immediate 
verbal  and  nonverbal  feedback  from  the  student  and  can  use  this  information  to 
tailor  their  interaction  with  the  student  based  on  prior  experiences.  In  an  automated 
training  environment,  the  intelligent  tutor  only  has  sensor  data  and  its  knowledge 
model  to  determine  which  strategies  are  good  at  motivating.  There  has  been 
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research  regarding  the  method  in  which  an  intelligent  tutor  can  support  student 
motivation  (Sheldon  2001);  however,  this  is  still  an  area  of  need. 

As  stated  by  Holden  et  al.  (2012;  p.  4),  “Increasing  computer-based  tutoring 
systems  (CBTS)  understanding  of  a  learner  can  foster  higher  levels  of  learning  by 
optimizing  instruction  based  on  the  learner’s  individual  needs.”  This  project  seeks 
to  develop  a  MAT  that  will  provide  an  automated  means  of  determining  a  student’s 
personal  motivators  and  selecting  the  optimal  motivational  strategy  for  an 
intelligent  tutor  to  employ  during  a  given  session.  Individualizing  the  external 
motivators  presented  to  a  student  has  the  potential  to  significantly  improve  the 
learning  rate  and  retention  of  the  trainee  (Isaksen  and  Ramberg  2005).  If  the  process 
of  determining  what  motivates  an  individual  student  to  engage  in  the  learning 
process  can  be  automated,  it  is  then  possible  to  develop  a  MAT  to  enable  the 
intelligent  tutor  to  administer  strategies  that  motivate  the  student  tailored  to  their 
unique  preferences.  The  efficacy  of  this  MAT  can  be  measured  by  the  increase  in 
effort,  attention,  and  learning  during  a  given  training  session. 

To  motivate  means  to  be  “moved  to  do  something”  (Ryan  and  Deci  2000). 
Motivation  is  essential  to  learning  (Keller  1987a).  When  in  a  motivated  state,  an 
individual  is  internally  inclined  to  initiate  a  task  and  persevere  throughout  its 
completion.  In  essence,  motivation  increases  an  individual’s  level  of  engagement 
(Magill  1980),  while  unmotivated  people  disengage  from  the  task  completely  or 
shut  down.  The  contributing  factors  of  motivation  is  based  on  an  individual’s  needs, 
values,  goals,  and  emotions  (Locke  2000)  on  the  reinforcer  and  task.  Motivation 
can  be  classified  into  2  types  (del  Soldato  and  du  Boulay  1999;  Kember  et  al.  1999; 
Noels  et  al.  1999):  1)  intrinsic  motivation,  which  refers  to  an  individual’s  internal 
desire  to  achieve,  and  2)  extrinsic  motivation,  which  refers  to  external  rewards  that 
encourage  an  individual  to  achieve.  While  intrinsic  motivation  is  an  innate  trait, 
extrinsic  motivation  is  a  state  that  an  instructor  or  automated  learning  environment 
can  influence. 

External  motivators  are  essentially  a  reward  that  reinforces  the  individual’s 
behavior  to  put  forth  an  effort  to  achieve  a  given  goal.  The  concepts  that  motivate 
an  individual  differ  and  could  include  the  potential  for  money,  recognition  (e.g., 
grades  or  award),  and  fun.  If  the  student  extends  an  effort  to  perform  well  in  the 
learning  environment,  those  motivators  become  rewards  upon  achievement  of  the 
learning  goal.  An  example  is  a  student  whose  primary  motivator  is  recognition.  If 
the  instructor  says  that  a  formal  announcement  of  the  student  receiving  the  highest 
test  score  will  be  made,  the  student  will  put  forth  effort  into  preparing  for  the  test. 
After  this  student  receives  the  highest  score  in  the  class,  as  a  result  of  this  effort, 
the  reward  would  be  having  the  teacher  announce  this  high  score  to  the  class.  In 
other  words,  the  motivator  can  be  likened  to  a  carrot  waived  in  front  of  a  rabbit  to 
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get  it  to  come.  After  the  rabbit  makes  the  effort  to  move  and  acquires  the  carrot,  it 
is  rewarded  by  getting  to  eat  the  carrot.  The  receipt  of  this  reward  is  a  reinforcement 
of  the  student’s  motivation,  which  results  in  motivating  the  student  to  perform  in  a 
similar,  subsequent  situation. 

The  concept  of  reinforcement  has  been  used  extensively,  dating  back  to  early 
classical  conditioning  experiments  (Pavlov  1927).  It  is  possible  to  motivate  by  use 
of  a  reinforcement  reward  to  increase  motivation  and  the  related  behaviors  that 
bring  the  individual  closer  to  achieving  a  goal  (French  1955;  Keller  1987a).  A 
reinforcer  is  the  stimulus  in  the  form  of  a  reward  or  incentive  that  is  received  after 
achievement  of  a  goal.  Consequently,  when  reinforcers  are  tailored  to  an 
individual’s  aspiration,  they  can  reinforce  motivation  and  the  desire  to  achieve  their 
goal  again. 

To  devise  an  automated  means  of  identifying  an  individual’s  motivator  and 
translate  this  motivator  to  the  real-world  context  within  training,  the  whole 
spectrum  of  motivation,  reinforcement  learning,  and  reinforcers  must  be 
scrutinized.  Consequently,  we  must  review  the  psychology  of  motivation.  The 
subjective  perspective  of  reinforcement  is  split.  Some  researchers  are  for  the  use  of 
reinforcers  and  others  are  cautious  because  of  the  complexity  of  humans.  The 
success  of  reinforcers  hinges  on  being  cognizant  of  types  of  motivation,  influential 
factors,  and  variations  of  implementation.  As  the  development  of  new  information 
about  the  nature  of  reinforcement  unfolds,  neuroscience  offers  an  alternative 
perspective  on  an  internal  view  of  what  happens  to  the  body  during  reinforcement. 
This  will  also  scope  the  side  of  learning  rate  and  retention,  as  well  as  individual 
differences  provided  by  literature.  Then  optimization  on  the  link  to  the  real-world 
integration  of  reinforcement  and  outcomes,  as  used  by  special  education  teachers, 
schools,  in  the  work  place,  and  marketing  to  influence  an  increase  specific 
responses.  Assessments  used  by  the  institutions  to  determine  the  optimal  reinforcer 
and  the  classification  of  tools  will  be  investigated.  Once  the  complete  spectrum  is 
laid  out,  careful  integration  and  alignment  of  reinforcement  strategies  can  be 
determined  for  individualized  or  personalized  training. 

A-3  Background  Research 

Individuals  are  driven  by  different  motivation.  Some  may  be  more  intrinsically 
motivated  and  others  extrinsically  motivating.  The  psychology  will  be  a  factor  in 
the  assessment  as  well  as  classifiers.  Motivation  is  complex  because  it  is  contextual 
and  domain  specific.  Social  cognitive  models  assess  specific  subject  area  such  as 
math  and  reading  and  reference  it  to  specific  teachers.  Some  subjects  need  more 
motivation  then  other  subjects.  Also,  a  person’s  individual  culture,  demographic, 
or  personality  characteristics  also  influence  motivation.  Students  can  be  motivated 
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in  multiple  ways  so  it  is  important  on  how  and  why  they  are  motivated  and  not  just 
labeling  as  motivated  and  not  motivated  (Linnenbrink  and  Pintrich  2002).  Social 
cognitive  models  look  at  4  key  social-motivational  theories:  self-efficacy, 
attributions,  intrinsic  motivation,  and  goal  orientations.  (Deci  et  al.  1999;  Eccles  et 
al.  1998;  Graham  and  Weiner  1996;  Pintrich  and  Schunk  2002).  They  are  closely 
associated  with  motivation  and  achievement. 

A.3.1  Behavior  Modification 

Humans  are  manipulated  by  reward  and  punishment;  those  experiences  help  shape 
and  guide  us  into  the  people  we  are  today.  This  manipulation  was  even  recognized 
as  early  as  the  1600s  when  Locke  (1632-1704)  famously  wrote,  “Good  and  evil, 
reward  and  punishment,  are  the  only  motives  to  a  rational  creature;  these  are  the 
spur  and  reins  whereby  all  mankind  are  set  on  work,  and  guided.”  There  is  an 
extensive  body  of  research  that  has  provide  evidence  that  human  behavior  can  be 
modified  though  the  use  of  reinforcement  provided  in  a  stimulus-response 
connection.  A  sustaining  reinforcement  will  intensify  the  bond  and  reoccurrence  of 
the  response.  If  the  reinforcement  is  displeasing,  the  response  will  decrease.  As 
stated  by  Thorndike’s  Law  of  Effect,  “The  greater  the  satisfaction  or  discomfort, 
the  greater  the  strengthening  or  weakening  of  the  bond”  (Thorndike  1911,  p.  244). 
Since  then  we  have  expanded  our  knowledge  on  reward  and  punishment  and  how 
to  begin  manipulating  motivation  in  a  person.  A  goal  of  this  project  is  to  use 
reinforcers  to  promote  student  motivation  to  learn.  The  following  provides  an 
overview  of  the  long  history  of  behavior  modification  research. 

One  of  the  most  famous  historical  investigations  into  behavioral  manipulation  is 
Pavlov’s  classical  conditioning  of  reflexes  (Pavlov  1927).  He  used  the  terminology 
unconditioned  stimulus  (US)  for  items  that  produced  an  unconditioned  response 
(UR).  He  demonstrated  that  it  was  possible  to  train  for  the  occurrence  of  the  same 
response  by  manipulating  the  US  with  a  conditioned  stimulus  (CS).  When  the  CS 
is  presented  with  the  US,  the  resultant  conditioned  response  (CR)  is  the  same  as  the 
UR.  However,  this  phenomenon  is  not  always  consistent.  Sometimes  expected 
responses  do  not  occur  or  happen  to  a  different  degree  with  the  conditioned 
response  as  with  the  stimulus  response.  The  US  also  can  result  in  additional 
responses  that  were  not  present  at  the  start  of  the  behavior  modification  effort.  To 
address  these  issues,  Thorndike  (1911)  investigated  animal  nonreflexive  behaviors 
and  how  they  can  be  changed. 

Modification  ensues  based  on  the  form  of  reinforcement  provided  in  a  stimulus- 
response  connection.  Thorndike’s  approach  is  that  learning  is  the  result  of  trial  and 
error  initially,  and  then  from  the  interconnectedness  of  satisfaction  and 
dissatisfaction  after  the  response  is  received.  One  of  his  most  famous  studies 
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involved  training  cats  to  efficiently  escape  from  a  puzzle  box.  He  found  that  cats 
learned  from  trial  and  error,  but  perceived  reward  when  they  were  able  to  escape 
from  the  puzzle  more  quickly.  This  decrease  in  the  amount  of  time  to  escape  is 
known  as  escape  latency.  Thorndike  attributed  the  decrease  in  escape  latency  due 
to  the  strengthening  of  the  stimulus  (shorter  route)-response  (decreased  time  in  the 
puzzle).  The  strengthening  of  a  bond  by  receiving  satisfaction  is  known  today  as 
positive  reinforcement. 

Expanding  on  the  work  of  Thorndike,  Skinner  (1911)  created  principles  of  learning 
by  voluntary  reinforcement.  This  reinforcement  looked  at  behaviors  beyond 
reflexes  known  as  operant  conditioning.  His  research  established  that  the  operant 
response  is  hinged  on  the  environment  to  produce  the  outcome.  As  in  contrast  to 
Thorndike’s  instrumental  conditioning,  Skinner  used  operant  learning  where  an 
animal  can  participate  in  reinforcement  a  multitude  of  times  during  one  session. 
This  was  assessed  by  the  response  rate  or  amount  of  time  reinforced  in  a  set  amount 
of  time.  He  established  3  parts  of  operant  conditioning:  1)  the  context  of  the 
response  (positive,  negative,  and  neutral),  2)  the  response,  and  3)  the  stimulus  that 
followed  the  response.  Skinner  defined  a  reinforcer  as  anything  that  magnifies  the 
frequency  of  the  operant  response. 

The  reinforcement  schedule  plays  an  essential  part  in  the  response.  There  are 
continuous  reinforcement  and  partial  reinforcement  schedules.  Continuous 
reinforcement,  providing  the  reinforcer  after  each  response,  has  a  fast  rate  of 
conditioning  but  also  a  fast  rate  of  extinction.  Once  the  outcome  is  achieved,  there 
is  a  post-reinforcement  pause  before  the  human/animal  continues  with  the  task. 
When  reinforcement  is  only  provided  after  a  certain  amount  of  responses  or  a  fixed 
ratio,  which  is  a  partial  reinforcement  schedule,  the  response  rate  is  quick. 
However,  the  extinction  rate  is  slower  than  when  a  continuous  reinforcement 
schedule  is  used.  With  partial  reinforcement,  a  variable  ratio  is  not  tied  to  specific 
number  of  responses  but  instead  varies  in  the  amount  of  time  until  the  reinforcement 
is  provided.  There  is  less  post-reinforcement  pause  because  reinforcement  could 
occur  at  any  moment.  When  the  amount  of  time  is  set,  it  is  called  a  fixed  interval 
reinforcement.  The  response  rate  is  slower  and  there  are  not  many  real-world 
situations  to  use  a  fixed  interval.  The  variable  interval  reinforcement  has  a  medium 
response  rate  and  a  slow  extinction  rate.  The  time  is  varied  before  reinforcement 
occurs.  Differential  reinforcement  rate  of  low  rates  (DRL)  is  given  once  a  certain 
amount  of  time  passes  over  the  previous  response.  The  differential  reinforcement 
of  high  rates  (DHR)  has  the  fastest  response  time  because  they  need  to  respond  so 
many  times  within  a  set  period  of  time.  (Ferster  and  Skinner  1957). 

Pavlov,  Thorndike,  and  Skinner  developed  and  tested  varying  approaches  to  using 
a  rewards  system  to  elicit  a  desired  behavior,  but  their  work  did  demonstrate 
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commonality  in  terms  of  reward  systems.  Specifically,  rewards  need  to  consider  the 
context  of  the  application  and  meaningfulness  to  the  individual,  or  animal,  whose 
behavior  is  being  modified.  In  developing  the  MAT,  understanding  the  context  and 
meaning  of  the  motivational  strategy,  as  well  as  the  reinforcer  presentation 
schedule,  will  be  important. 

A.4  Psychology  of  Reinforcement 

Researchers  in  psychology  remain  conflicted  and  inconsistent  regarding  the 
effectiveness  of  reinforcers.  Some  researchers  are  concerned  with  using  reinforcers 
because  there  is  the  potential  that  reliance  on  the  reinforcer  may  result  in  a  decrease 
of  the  person’s  natural  drive  for  learning.  Other  researchers  support  the  use  of 
reinforcers  due  to  benefits  they  bring  and  that  they  have  minimal  consequences, 
and  yet  others  stand  that  rewards  are  complex. 

A-4.1  Rewards 

To  discuss  rewards  with  respect  to  motivation,  it  is  important  to  review  the 
distinction  between  extrinsic  and  intrinsic  motivation,  as  briefly  discussed  in  the 
Section  A-2.  A  person’s  intrinsic  motivation  is  seen  as  the  greatest  motivational 
drive  for  a  person  because  satisfaction  and  the  basic  needs  of  a  person  are  met 
internally.  Intrinsic  motivation  is  fostered  through  autonomy  and  a  feeling  of 
competency.  There  are  3  subcategories  of  an  individual  with  intrinsic  motivation: 
they  seek  to  know,  accomplish,  and  be  stimulated  (Vallerand  et  al.  1992).  When  a 
trainee  is  intrinsically  motivated  to  know,  they  look  to  explore  and  are  curious.  A 
trainee  has  pleasure  and  stagflation  exploring,  learning,  and  understanding  at  the 
same  time.  In  a  state  of  intrinsic  motivation  toward  accomplishments,  trainees  feel 
satisfaction  from  accomplishment  or  creating  something.  The  last  intrinsic 
motivation  is  for  stimulation.  This  is  when  a  person  engages  for  pleasure 
stimulation  of  the  senses  such  as  fun  and  excitement.  They  seek  to  master  training 
techniques. 

External  motivation  seeks  to  instill  motivation  outside  of  a  person’s  own  interest, 
such  as  a  reinforcer.  There  are  different  types  of  extrinsic  motivation.  In  varying 
degree  from  more  controlling  to  least  they  are  external  regulation,  introjected 
regulation,  and  identified  regulation  (Deci  and  Ryan  2000).  In  external  regulation, 
a  person  is  motivated  by  the  reward.  Introjected  regulation  occurs  when  the 
individual  wants  to  keep  their  own  expectations  and  avoid  guilt  even  if  it  isn’t 
valued  or  seen  as  fun.  Identified  regulation  is  the  person  value  of  the  behavior,  but 
still  does  not  like  it  (Deci  and  Ryan  2000).  Studies  have  shown  a  reduction  of 
intrinsic  motivation,  even  in  an  individual  with  high  intrinsic  motivation,  when  an 
extrinsic  motivation  strategy  is  used  (Deci  et  al.  1999;  Lepper  et  al.  1973;  Tang  and 

Approved  for  public  release;  distribution  is  unlimited. 

65 


Hall  1995;  Frey  and  Jengen  2001).  Using  an  extrinsic  motivator  for  a  highly, 
intrinsically  motivated  individual  also  can  result  in  a  decrease  of  their  creativity 
(Kohn  1999;  Pink  2011).  Extrinsic  motivators  can  be  seen  as  controlling  to  the 
individual  who  is  intrinsically  motivated.  Consequently,  there  is  a  wide  range  of 
responses  for  an  individual  who  feels  the  external  reward  is  controlling.  This 
expression  can  appear  as  acceptance,  impassiveness,  resistance,  or  resentment  due 
to  individual  differences.  Examples  of  external  motivators  that  have  been  observed 
to  decrease  an  individual’s  intrinsic  motivation  are  threats  (Deci  and  Cascio  1972), 
timelines  (Amabile  et  al.  1976),  and  deadlines  (Reeve  and  Deci  1996). 
Consequently,  if  the  external  motivator  is  seen  as  controlling,  it  will  undermine  an 
individual  exhibiting  intrinsic  motivation. 

In  contrast,  some  researchers  found  rewards  have  an  increase  in  creativity 
(Eisenberger  and  Aselage  (2009)  and  positive  correlates  to  quantity  and  quality  in 
performance  (Cerasoli  et  al.  2014;  Garbers  and  Konradt  2014)  when  accompanied 
with  positive  feedback  and  choice  (Byron  and  Khazanchi  2012).  The  counter 
argument  is  negative  effects  from  rewards  are  minimal  when  presented  and  applied 
correctly  (Cameron  et  al.  2001).  They  found  an  increase  in  task  interest  for  rewards 
for  finishing  a  task,  each  unit  solved,  surpassing  a  score,  and  exceeding  others 
(Cameron  et  al.  2001).  In  comparison,  Deci  et  al.  (1999)  found  a  decrease  on  task 
completion  and  no  effect  on  completion  performance.  Research  has  confirmed  that 
reinforcers  are  ineffective  in  instances  that  learning  is  of  high  interest,  tangible 
rewards  are  offered  beforehand,  rewards  are  given  without  success,  or  rewards  are 
not  dependent  upon  performance  (Cameron  et  al.  2001).  More  recently,  Hendijani 
etal.’s  (2016)  experiment  found  both  intrinsic  and  extrinsic  rewards  increased 
performance  and  performance  depending  reward  (money)  had  a  positive  effect  on 
motivation  and  performance  no  matter  a  person’s  intrinsic  behavior. 

Thomas  and  Jansen  (1996)  propose  4  types  of  intrinsic  rewards:  1)  sense  of 
meaningfulness,  2)  choice,  3)  competence,  and  4)  progress.  However,  their 
definition  of  intrinsic  and  extrinsic  differ  from  those  previously  discussed. 
Specifically,  they  define  intrinsic  motivation  as  referring  to  the  anticipation  of  the 
psychological  rewards  an  individual  derives  from  performing  a  task,  and  extrinsic 
motivation  refers  to  the  anticipation  of  task  contingent  rewards  such  as  money, 
recognition,  and  praise. 

There  has  been  research  regarding  the  role  of  instructional  feedback  on  student 
motivation  and  performance  (Trimmers  et  al.  2013).  There  are  different  types  of 
instructional  feedback,  such  as  1)  knowledge  of  results  (KR),  in  cases  where  student 
is  incorrect,  2)  knowledge  of  correct  response  (KCR),  and  3)  elaborated  feedback 
(EF).  Trimmers  et  al.  (2013)  report  that  KCR  and  EF  are  associated  with  improved 
learning  outcomes,  although  the  provision  of  this  type  of  feedback  does  not  always 
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guarantee  learning  success.  An  explanation  is  that  KCR  and  EF  are  likely  perceived 
by  the  student  as  positive  feedback,  and  it  has  been  demonstrated  the  positive 
instructional  feedback  leads  to  improved  learning  outcomes  (Sheldon  2001). 
Further,  Trimmers  et  al.  (2013)  conducted  a  study  that  demonstrated  a  positive 
relationship  between  a  student’s  feedback  seeking  behaviors  and  their  self-reported 
motivation  as  assessed  with  the  online  motivational  questionnaire.  Consequently, 
for  some  students,  instructional  feedback  may  be  perceived  as  a  type  of  reward. 

A-4.2  Goals 

Goal  setting  is  a  motivational  strategy  to  increase  a  behavior.  In  the  achievement 
goal  theory,  there  are  2  goals:  mastery  and  performance  (Ames  1992;  Elliot  and 
Harackiewicz  1996;  Elliot  1997;  Elliot  and  Church  1997;  Harackiewicz  et  al.  1998). 
Mastery  goals  look  at  “developing  new  skills,  trying  to  understand  their  work, 
improving  their  level  of  competence,  or  achieving  a  sense  of  mastery  based  on  self- 
referenced  standards”  (Ames  1992,  p.  262).  Performance  goals  focus  on  self-worth 
and  ability.  It  sets  the  expectations  on  the  level  of  performance  the  individual  needs 
to  achieve.  Goals  are  more  effective  when  they  are  set  at  the  right  level  of  obtainable 
objectives  that  challenge  the  individual  based  on  their  past  experiences  (Magill 
1980).  Proper  matching  of  a  reinforcer  to  a  goal  has  been  found  to  reduce 
frustration.  Setting  the  right  type  of  goals  for  the  individual  is  important.  Elliot  et 
al.  (2011)  created  the  3x2  achievement  model  of  goals  that  use  a  questionnaire  to 
understand  the  individual’s  needs  with  respect  to  goal  setting.  The  3  in  the  matrix 
is  composed  of  3  goal  types:  1)  task-based,  2)  self-based,  and  3)  other.  The  2  in  the 
matrix  is  competence  valence,  which  can  either  be  positive  or  negative.  The 
questionnaire  evaluates  the  individual’s  view  of  the  task,  self,  and  other 
interpersonal  skills,  and  the  way  in  which  those  views  combine  with  valence  of 
positive  or  negative.  Task-based  goals  are  referred  to  as  mastery  of  the  task,  such 
as  task  comprehension  and  percent  correct.  Self-based  goals  are  a  reflection  the 
competency  a  person  feels  they  possess,  for  example,  if  a  person  feels  successful 
or  poorly  about  the  task  compared  to  prior  and  future  experiences.  Other  goals 
compare  the  individual’s  competency  to  others’  performance.  As  with  basic 
instincts  of  the  environment,  motivation  results  in  an  individual  either  approaching 
(fight)  or  withdrawing  (flight)  from  a  goal,  as  associated  with  positive  and  negative 
valence  respectively.  The  task  approach  goal  is  viewed  as  whether  the  task  is 
completed  correctly,  whereas  the  task  avoidance  looks  at  avoiding  completion  of 
the  task  incorrectly.  The  self-approach  goal  deals  with  performing  better  or  learning 
more  than  previously  achieved,  whereas  the  self-avoidance  goal  focused  on  not 
performing  worse  or  learning  less  than  previously  achieved.  The  other-approach 
goal  seeks  for  the  individual  to  do  better  than  others,  where  the  other- avoidance 
goal  aims  at  a  person  not  doing  worse  than  others  (Elliot  et  al.  2011).  Therefore,  it 
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is  important  to  note  that  the  goal  for  an  individual  is  related  to  and  is  impacted  by 
their  specific  motivation,  and  thus,  supports  that  goals  need  to  be  framed  to  address 
individual  differences.  When  setting  individual  goals  based  on  the  individual,  it 
helps  the  individual  meet  their  need  for  motivation.  One  Warfighter  strives  to  out 
preform  others  and  their  goal  will  be  written  as  an  approach  goal.  The  focus  of  their 
motivation  is  to  advance  passed  others.  Avoidance  goals,  is  written  for  those  who 
have  more  of  a  social  anxiety  and  are  motivated  based  on  performance  versus  others 
They  do  not  want  to  appear  to  have  failed  others.  In  wartime,  trainees  are  motivated 
by  different  reasons — either  to  be  the  best  or  to  not  do  worse  than  another  peer. 
When  goals  are  written  based  on  their  specific  motivation  as  an  individual  they  can 
relate  to  the  reason  why  they  are  working  for  that  goal. 

A-5  Neuroscience  of  Memory  Rewards 

The  term  reward  or  reinforcer  in  neuroscience  includes  a  spectrum  of  items 
including  tangibles,  social,  motivation  associated  with  positive  and  negative 
effects,  and  moving  away  from  or  toward  a  stimulus  (Harmon- Jones  et  al.  2013). 
The  parts  that  control  decision  making  and  goal-related  behavior  (Fareri  et  al.  2008; 
Schultz  2007)  are  in  the  neural  circuitry,  which  is  divided  into  a  connection  of  the 
cortical  (executive,  regulatory)  and  subcortical  (primitive  and  emotion  areas) 
regions  (Carlson  et  al.  2011;  Ernst  and  Spear  2009;  Fareri  et  al.  2008;  Knutson  et 
al.  2003;  Mayes  et  al.  2009).  Rewards  are  addressed  by  the  basal  ganglia,  and  more 
specifically,  the  striatum  and  the  prefrontal  and  orbitofrontal  cortices  and  their 
connections  of  corticostriatal  circuits  or  loops.  When  positive  reinforcement 
occurs,  the  brain  is  rewarded  with  an  increase  of  dopamine  (DA).  DA  is  released  in 
the  midbrain  in  an  area  called  the  ventral  striatum,  and  more  specifically,  the  tiny 
nucleus  of  closely  inhabited  neurons  called  accumbens.  The  discharge  provides  a 
short-term  effect  on  affect,  but  the  activation  of  the  DA  neurons  is  thought  to  begin 
reinforcement  learning.  The  DA  release  occurs  from  unexpected  rewards  as  well  as 
anticipated  rewards,  and  DA  emission  decreases  when  the  reward  is  not  given.  The 
DA  discharge  also  occurs  from  the  expectation  and  value  of  the  reward  held  by  the 
individual.  The  interplay  of  the  DA  and  the  reward  related  process  (RL)  is  not  fully 
understood  (Howard-Jones  and  Jay  2016).  However,  the  RL  is  thought  to  be  in  the 
ventral  tegmental  area  (VTA),  which  releases  the  DA  to  various  parts  of  the  brain 
such  as  the  prefrontal  cortex  (PFC),  nucleus  accumbens  (NAc),  amygdala,  and 
hippocampus  (Barrot  2014).  Rewards  also  have  psychological  parts  of  liking, 
wanting,  and  learning  that  can  be  conscious  or  nonconscious.  The  firing  of 
dopaminergic  neurons  may  signal  the  want  in  reaching  goals  (Berridge  2012; 
Bromberg-Martin  et  al.  2010)  and  begin  the  seeking  of  the  goal  (Alcaro  and 
Pankespp  2011).  The  DA  neurons  act  in  both  a  phasic  (burst  and  firing)  and  a  tonic 
(slow  and  irregular)  manner  that  is  thought  to  be  involved  with  motivational  state 
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(Ikemoto  and  Tan  2015).  Phasic  firing  responds  to  reward  availability  (Schultz 
2007,  2010)  and  tonic  dopamine  responds  to  novelty  (Niv  et  al.  2007)  and  an  overall 
level  of  motivation  (Ikemoto  2007;  Kaplan  and  Oudeyer  2007).  Reward  prediction 
error  (RPE;  Ernst  and  Spear  2009)  is  taken  from  the  discrepancies  between  the 
projected  worth  of  future  rewards  or  punishments  and  their  actual  value.  From  past 
experiences,  the  brain  makes  predictions  on  the  best  way  to  react  and  the  anticipated 
outcome.  Learning  happens  when  the  predicted  outcome  is  different  than  the  actual 
outcome  and  creates  an  RPE.  This  RPE  has  shown  consistent  increase  in  the  tonic 
levels  from  rewards  and  is  maximized  with  the  uncertainty  of  rewards  (Fiorillo  et 
al.  2003;  Preuschoff  2006;  Tobler  et  al.  2007). 

Similarly,  a  mental  model  is  also  dependent  on  previously  learned  skills  and 
experiences  and  how  it  was  obtained.  A  mental  model  helps  us  manipulate 
problems  to  find  difficult  and  abstract  answers.  False  mental  models  can  happen 
when  trainees  are  in  discovery  learning  (Briggs  1990)  and  practicing.  Overlearning 
materials  helps  with  creating  good  mental  models  and  helps  with  retention. 
However,  persistence  is  necessary  to  continue  repeated  practice  (Driskell  et  al. 
1992).  Rumelhart  et  al.  (1986)  found  that  humans  need  to  use  cognitive  schematas 
to  find  patterns,  modeling  the  world  (anticipation  of  a  new  state  from  actions  they 
see).  A  learner  uses  their  experiences  to  manipulate  data  and  think  logically  in  math 
problems  and  science. 

Memory  goes  through  the  following  steps:  encoding,  storing,  and  retrieving.  There 
are  3  memory  systems:  1)  short-term  memory,  2)  working  memory,  and  3) 
long-term  memory.  The  short-term  memory  stores  the  information  for  brief  periods 
of  time,  whereas  the  working  memory  processes  and  uses  the  information.  The 
working  memory  has  the  following  types:  auditory  memory,  visual-spatial 
memory,  and  the  episodic  buffer,  which  is  linked  to  long-term  memory.  The  visual 
working  memory  shares  parts  and  interacts  with  attention  (Gazzaley  and  Nobre 
2012;  Zanto  et  al.  2011).  Short-term  memory  is  limited  in  its  importance  to 
motivation,  and  therefore,  does  not  add  value  to  determining  a  reinforcer.  However, 
working  and  long-term  memory  are  relevant,  as  detailed  in  the  following  sections. 

A-5.1  Working  Memory 

Individuals  are  motivated  to  increase  their  attention  to  receive  a  reward,  as  shown 
by  research.  Monetary  rewards  show  a  positive  effect  on  attentional  performance 
and  response  rate  (Small  et  al.  2005;  Englemann  et  al.  2009).  Specifically, 
Englemann  et  al.  (2009)  used  face  pictures  that  were  difficult  to  see,  easy  to  see, 
and  noise.  Participants  received  extra  money  for  correct  responses  that  was  on  a  pie 
chart,  and  had  various  reward  magnitudes  and  valences.  The  results  showed  that 
performance  increased  as  a  result  of  incentive.  In  the  Small  et  al.  (2005)  study, 
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participants  were  asked  to  respond  to  x’s  and  not  +’s  in  a  specific  amount  of  time. 
One  group  was  given  money  that  they  could  lose  if  they  didn’t  meet  the  time,  while 
the  other  group  could  earn  money  for  responding  fast.  They  looked  at 
disengagement  and  visual  spatial  expectancy.  The  evidence  suggested  that  penalties 
and  rewards  influence  attention.  The  brain  was  activated  in  different  areas  based  on 
winning  and  losing.  The  orbitofrontal  cortex  (OFC)  was  more  engaged  when 
winning  and  the  dorsal  anterior  cingulate  cortex  was  more  engaged  with  losing 
(Small  et  al.  2005). 

Some  research  has  also  shown  that  rewards  can  help  an  individual  to  ignore 
distractors  (Della  Libera  and  Chelazzi  2009).  They  found  that  this  ability  to  ignore 
distractors  is  shaped  by  past  encounters  of  the  objects  and  consequences.  Once  the 
participants  learned  that  objects  were  the  target  versus  distractors,  it  was  difficult 
for  them  to  switch  the  target  to  a  distractor.  Rewards  may  also  enhance  performance 
when  looking  for  the  target  with  multiple  distractions  (Chun  et  al.  2011).  Chelazzi 
and  colleagues  (2013)  eluded  that  rewards  potentially  act  as  “teaching  signals  for 
learning”  and  enhancing  attentional  operations  such  as  selecting  or  ignoring.  The 
rewards  could  lead  to  faster  and  better  visual  performance.  They  also  found  that 
rewards  increase  areas  of  importance  on  visual  stimuli  and  also  suppress  distracting 
stimuli  (Chelazzi  et  al.  2013.)  In  another  study,  Anderson  et  al.  (2011,  2013)  paired 
a  high  and  low  reward  with  a  color  and  were  able  to  train  the  participants  to  view 
specific  colors  as  a  visual  distractor.  The  results  demonstrated  that  the  value 
associated  with  the  previously  awarded  stimuli  interfered  with  performance.  This 
provides  evidence  that  the  reward  shaped  the  attentional  signal.  Chelazzi  and 
colleagues  (2014)  placed  high  rewards  and  low  rewards  for  correct  responses.  In 
this  study,  participants  could  earn  the  high  or  low  reward  by  a  specific  location  in 
one  group,  and  they  did  not  receive  any  rewards  for  the  task  in  the  second  group. 
Participants  were  asked  to  find  targets  on  the  screen.  In  the  rewards  group,  the 
participants  viewed  finding  targets  in  the  high  reward  section  as  a  higher  priority 
than  finding  the  target  in  a  low  reward  section.  The  researchers  determined  that 
reward  based  learning  can  change  the  priority  of  a  special  task.  That  result  was  also 
found  several  days  after  the  learning  and  was  generalized  to  a  new  task.  Those 
studies  show  the  strong  connections  between  attention  and  reward,  and  also  how 
attention  is  connected  to  the  working  memory. 

Gong  and  Li  (2014)  found  that  learned  rewards  do  improve  the  visual  working 
memory  in  3  experiments.  The  first  experiment  demonstrated  that  high  reward 
(money)  improves  detection  sensitivity  over  low  reward.  The  second  experiment 
showed  that  when  replacing  a  monetary  reward  with  a  feedback  reward,  they  did 
not  see  the  difference  in  sensitivity  detection  observed  in  the  first  experiment.  The 
third  experiment  showed  an  increase  in  performance  for  the  visual  working  memory 
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when  it  was  high  reward  versus  low.  Long-term  memory  is  composed  of  2  parts: 
declarative  and  procedural.  Research  has  shown  that  rewards  affect  declarative 
memory,  which  is  a  part  of  our  long-term  memory  that  helps  us  remember 
experiences,  facts,  and  verbal  knowledge.  Two  categories  of  declarative  memory 
are  the  episodic  (personal  experiences)  and  semantic  memory  (factual  information). 
When  presented  with  a  monetary  reward  to  recall  visual  scenes,  the  group  who  was 
given  a  greater  reward  had  a  higher  recall  of  the  scenes  (Adcock  et  al.  2006).  This 
suggests  that  visual  working  and  long-term  memory  can  be  improved  by 
manipulating  attention  with  rewards. 

The  medial  temporal  lobe  (MTL)  system  plays  a  role  in  creating  and  condensing 
memories,  but  also  transferring  new  things  learned  (Cohen  et  al.  1997;  Eichenbaum 
2004;  Nadel  and  Moscovitch  1997;  Squire  1992).  There  is  also  a  growing  body  of 
research  suggesting  the  hippocampus,  which  is  part  of  the  MTL  system,  sorts  out 
relationships  of  the  internal  states  and  the  external  environment  into  a  person’s 
motivation  and  memory  that  guides  behavior  (Baudonnat  et  al.  2013).  Noradrenic 
systems  are  active  in  positive  and  negative  reinforcement  of  behavior  (Sara  2009). 
Noradrenaline  also  has  a  role  in  memory  formation  in  the  long-term  potentiation 
(LTP)  in  the  amygdala  and  the  hippocampus  (Mather  et  al.  2015;  Sara  2009).  The 
LTP  is  a  model  for  learning  and  memory  that  is  divided  into  early  and  late  stages. 
The  early  stage  is  short  term,  minutes  and  hours,  and  the  later  stage  is  where 
memories  become  more  permanent.  The  late  stage  requires  DA  for  encoding 
(Lisman  2011).  Serotonin  is  also  interconnected,  but  does  not  seem  to  play  a  role 
in  memory  formation  (Thomas  2015;  Werlen  and  Jones  2015).  However,  when 
serotonin  is  decreased,  impairment  encoding,  recall,  and  verbal  learning  occurs, 
although  it  does  not  impede  spatial  or  declarative  memory  (Mendelsohn  et  al. 
2009). 

A-5.2  Long-Term  Memory 

Studies  have  shown  that  rewards  make  an  instant  influence  (Wolosin  et  al.  2012). 
Other  studies  show  no  immediate  influence,  but  yet  a  long-term  effect  because  of 
the  need  to  consolidate  (Adcock  et  al.  2006)  the  impact  of  the  rewards.  For  example, 
Murayama  and  Kuhbandener  (2011)  conducted  a  study  in  which  participants 
answered  trivia  questions  where  one  group  received  a  monetary  reward  for  correct 
responses  while  the  other  group  received  no  reward.  After  10  min,  both  groups 
showed  no  difference  recalling  answers,  but  after  a  week,  the  group  that  was 
rewarded  recalled  more  information  than  the  nonrewarded  group.  However,  they 
observed  that  if  the  individual  perceived  the  question  as  interesting,  the  monetary 
reward  did  not  increase  performance.  This  supports  extrinsic  rewards  only 
influence  a  person’s  motivation  if  the  task  is  not  intrinsically  motivating.  In  2013, 
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Murayama  and  Kitagami  demonstrated  extrinsic  reward  enhanced  memory 
separately  from  a  person’s  attention  and  motivation.  They  concluded  that 
dopaminergic  memory  consolidation  was  an  effect  of  extrinsic  rewards. 

Rewards  given  with  social  interaction  has  also  been  investigated.  There  is  overlap 
in  the  neural  circuits  and  the  mesolimbic  brain  region  for  social  cues  and  reward 
processes  (Knutson  and  Wimmer  2007).  One  study  completed  by  Izuma  et  al. 
(2008)  looked  at  the  social  aspect  of  having  a  positive  reputation  as  a  reward.  For 
this  study,  money  could  be  earned  and,  on  a  secondary  task,  participants  received 
feedback  about  their  personality.  Both  types  of  rewards  activated  the  same  brain 
area,  which  suggests  that  feedback  triggers  the  same  area  of  the  brain  as  monetary 
incentives.  In  another  study,  participants  had  to  decide  whether  to  donate  money  to 
an  organization  or  to  keep  the  5  dollars.  Some  of  the  participants  had  observers 
watching  what  they  decided  while  others  had  no  one  observing  them  (Izuma  et  al. 
2010).  The  charitable  organizations  were  categorized  by  value.  When  the  charity 
was  considered  as  high  value,  the  participant  would  donate  with  or  without  the 
observer  present.  If  the  charity  was  valued  low,  the  participant  would  not  donate 
even  with  the  observer.  When  the  charity  was  of  a  moderate  value,  the  participants 
were  not  consistent  in  deciding  whether  to  donate  or  keep  the  money.  Functional 
magnetic  resonance  imaging  (fMRI)  showed  that  activities  for  decision  making 
were  dependent  upon  observer  presence.  This  shows  that  social  approval  activated 
the  same  location  that  monetary  rewards  did  (Izuma  et  al.  2010).  Another  group  of 
researchers  also  looked  at  the  effect  on  giving  to  a  charitable  donation  versus 
keeping  money  for  themselves.  The  results  showed  that  donating  led  to  higher 
activity  in  the  ventral  striatum  then  receiving  money  (Moll  et  al.  2006).  This  may 
be  due  to  the  perceived  social  impact  of  donating  the  money. 

A-5.3  Summary 

Every  person  is  different  and  memory  enhancement  from  rewards  might  be 
dependent  upon  an  individual’s  value,  selectivity,  and  prior  experiences  with 
rewards  that  leads  them  to  be  satisfied.  Individuals  are  unique  down  to  our  genetic 
makeup.  It  has  been  observed  that  there  are  differences  in  performance  based  on 
the  amount  of  DA  released.  Small  amounts  of  DA  have  seen  an  increase  in 
performance  and  large  amounts  of  DA  resulted  in  a  decrease  of  performance 
(Wittmann  2011).  An  inverted  U  appears  from  the  dose  of  reward  given  (Apiitz 
2013)  and  individuals  fall  at  different  places  for  the  baseline.  The  literature  is  not 
very  extensive  on  how  rewards  (reinforcers)  affect  a  person’s  memory.  As  stated 
by  Miendlarzewska  et  al.  (2016,  p  157),  “To  our  knowledge,  this  is  the  first  effort 
to  integrate  the  far-reaching  effects  of  reward  motivation  on  learning  that  span 
phenomena  never  considered  before,  including-value  generalization  through 
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perceptual  similarity  or  various  forms  of  associative  learning,  as  well  as  the  impact 
of  reward  motivation  transfer  in  memory  recollection  and  transformation.”  They 
also  mention  that  there  needs  to  be  more  research  completed  with  individual 
personality  traits,  such  as  questionnaires  and  the  linkage  to  dopamine  and  memory. 
This  project  may,  therefore,  investigate  the  linkage  between  memory  and  brain 
activities  governed  by  dopamine,  as  well  as  the  adrenaline/noradrenaline  system 
and  the  process  to  use  this  linkage  to  determine  the  best  intervention  for  increasing 
their  motivation. 

A-6  Education 

The  best  source  of  current  practices  for  determining  a  motivational  tool  matched  to 
a  learner  comes  from  special  education.  Special  education  teachers  acknowledge 
that  a  student’s  motivation  is  maximized  by  different  outputs.  Reinforcement 
inventories  guide  motivator  selection.  Correct  identification  of  motivators  for 
individuals  has  been  proven  to  increase  goal  attainment  with  students  who  are 
autistic  or  have  other  social  and  emotional  deficits.  Some  students  apply  more  effort 
if  they  receive  peer  approval.  Others  feel  the  need  for  adult  approval.  Tangibles  are 
defined  as  things  they  can  hold  such  as  food,  money,  prizes,  and  stickers.  Others 
thrive  on  competition.  Many  want  free  time  away  from  the  task  or  the  use  of 
technology  for  entertainment. 

Applied  behavior  analysis  (ABA)  has  been  used  with  students  who  are  autistic  to 
evaluate  their  responses  to  different  motivational  strategies.  ABA  continuously 
monitors  and  graphs  the  effectiveness  of  motivational  strategy  (operant  behavior) 
on  the  student’s  motivation  (respondent  behavior),  and  the  results  are  used  to 
modify  the  motivational  strategy.  Reinforcer  inventories  are  used  to  find  the  value 
of  reinforcers  for  individuals  based  on  things  they  prefer  and  do  not  prefer.  Operant 
behavior  is  selected  based  on  the  results  of  a  history  of  consequences  and  it  is 
maintained  by  the  consequence  the  behavior  brings.  Reinforcers  are  preferred 
events  if  they  increase  the  chance  for  the  behavior  to  occur  again.  The 
reinforcement  from  the  reinforcers  can  be  either  positive  or  negative  reinforcement. 
Negative  is  seen  as  escape  and  avoidance.  For  example,  a  person  wants  a  break 
away  from  a  difficult  task  if  it  is  perceived  as  a  negative  reinforcement. 

There  are  different  types  of  rein  forcers  (Cooper  et  al.  2007;  Newman  2003). 
Primary  reinforces  are  naturally  occurring,  such  as  food,  water,  sleep,  and 
temperature;  and  secondary  reinforcers  are  paired  with  a  primary  reinforcer,  such 
as  food  provided  with  a  token.  Secondary  reinforcements  after  repeated  pairing 
with  the  primary,  becomes  a  primary  reinforcer  (e.g.,  grades,  money,  praise, 
strength,  rank).  The  identification  of  reinforcers  that  motivate  a  person  is  important 
for  its  effectiveness.  Methods  for  determining  the  optimal  reinforcer  in  educational 
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settings  include  directly  asking  the  student,  observing,  using  the  Premack  principle, 
sampling,  or  reinforcer  preference  assessments  (Ryan  2011).  The  Premack 
principle  refers  to  cause  and  effect  relationships  (e.g.,  if  you  do  this,  then  this  will 
happen),  such  as  if  a  child  gets  an  A  on  the  test,  then  they  can  ride  their  bike.  A 
reinforcer  or  preference  assessment  evaluates  a  student’s  preferences.  When  using 
a  reinforcer,  avoid  satiation  (when  reinforcers  no  longer  are  effective)  by  using  a 
selection  of  reinforcers  and  small  amounts.  Only  allow  the  reinforcer  to  be  given 
once  the  target  is  met  and  vary  the  pace  of  the  reinforcer.  Vary  the  tone  of  voice 
when  feedback  is  given  with  the  reinforcer  and  use  specific  and  varied  feedback  to 
make  it  fun  (Ryan  2011). 

Special  education  teachers  also  conduct  functional  behavioral  assessments  (FBAs) 
to  create  individual  behavioral  intervention  plans  (BIPs).  FBAs  are  the  most 
effective  intervention  for  changing  behavior  in  the  classroom  (Ingram  et  al.  2005; 
Blood  and  Neel  2007).  The  plan  begins  by  developing  a  goal  that  is  observable  and 
measurable  with  no  “fuzzy”  terminology  (Mager  1972).  The  success  is  driven  by  a 
clear  hypothesis  that  has  3  parts:  antecedent,  behavior,  and  function  of  the  behavior. 
Then  execution  steps  are  developed.  Reinforcing  the  individual  antecedent  is 
important  for  success  of  the  interventions.  A  flowchart  is  created  for  identifying  the 
problem,  which  takes  one  of  2  directions:  either  a  student  is  avoiding  something 
(attention,  unwanted  task  or  item,  sensory  input)  or  desires  something  (attention, 
activity  or  item,  or  sensory  input;  Taylor  and  Abernathy  2016).  In  the  case  of 
avoidance,  replace  the  behavior  with  something  acceptable.  To  do  this,  remove  or 
add  a  desired  task  (negative  reinforcement)  or  object  (positive  reinforcement).  If 
they  seek  attention  or  activity,  weaken  the  problem  by  removing  any  reinforcement 
of  negative  behavior  (extinction)  or  take  away  desired  task  (response  cost). 

Another  method,  positive  behavioral  supports  (PBSs),  originally  only  applied  to 
students  with  extreme  social  disabilities,  has  been  made  available  for  all  students 
to  better  optimize  learning.  PBS  has  its  foundations  in  operant  conditioning,  in 
which  students  are  rewarded  for  positive  behaviors.  When  PBS,  rules  and 
expectations  need  to  be  clear,  and  the  rewards  system  is  based  on  the  individual 
student’s  interests.  It  is  recommended  with  PBS  that  reward  systems  include  praise 
or  feedback,  as  well  as  a  token  economy  (McKevitt  et  al.  2012). 

A-7  Workplace 

Rewards  in  the  workplace  are  also  seen  to  increase  performance.  Compensation  and 
incentives — individual,  competitive,  or  cooperative — have  been  used  by 
companies  to  improve  performance,  behavior,  and  motivation  (Guzzo  et  al.  1985). 
People  will  work  for  rewards  and  to  avoid  punishment,  and  will  neglect  what  is 
neutral  (Kerr  1975).  In  the  workforce,  it  is  important  to  align  the  rewards  with  goals 

Approved  for  public  release;  distribution  is  unlimited. 

74 


and  outcomes  (Kerr  and  Slocum  1988).  When  developing  a  reward  plan,  it  is 
important  to  keep  in  mind  the  benefit  to  the  employee  when  they  reach  the  desired 
outcome  (Rothwell  and  Kanzanas  2008).  This  is  also  the  first  question  that 
instructional  designers  ask  when  creating  a  reward  system.  For  the  system  to  work, 
a  person  must  be  able  to  reach  success  and  feel  competent,  they  must  believe  they 
will  actually  receive  a  reward,  and  they  must  value  the  reward.  As  in  education  and 
psychology,  rewards  need  to  be  designed  intentionally,  controlled  by  others,  and 
understand  the  reward-achievement  relationship.  Instructional  designers  see  4  goals 
for  rewards:  attracting  people  to  an  organization,  keeping  people  with  the 
organization,  modifying  behavior  a  certain  way  or  increasing  creativity,  and 
increasing  achievement  for  work  results.  They  classify  rewards  into  2  categories: 
monetary  and  nonmonetary  incentives  (Rothwell  and  Kanzanas  2008). 

The  workplace  recognizes  that  individuals  respond  differently  to  incentives 
(Wageman  and  Baker  1997).  As  seen  in  the  workplace,  individuals  who  work 
independently,  competitiveness  is  seen  as  an  effective  incentive.  Alternatively, 
cooperation  incentives  are  effective  group  tasks  (Zingheim  and  Schuster  2000; 
Wageman  2006).  The  reward  needs  to  fit  the  individual,  and  their  attitudes  and 
inclinations.  Money  has  a  proven  positive  effect  as  a  reward  in  the  United  States 
and  other  countries  (Du  and  Choi  2010).  Monetary  rewards  can  increase 
productivity  by  30%  (Locke  et  al.  1980).  However,  the  workplace  is  also  cognizant 
of  how  external  rewards  can  decrease  intrinsic  motivation  (Eisenberger  and 
Cameron  1996).  Large  amounts  of  money  can  fail  to  motivate  (Beer  and  Cannon 
2004),  they  can  lead  to  a  decline  in  performance  for  fear  of  failure  (Chib  et  al.  2013) 
and  increase  expectations  on  payouts  (Beer  and  Cannon  2004).  Bari  et  al.  (2013) 
found  that  nonfinancial  rewards  can  increase  attitude  and  performance  in  the 
workplace.  They  found  a  positive  impact  on  feedback,  freedom,  career 
development  plan,  valuation  of  employees,  learning  programs,  open  and 
comfortable  work  environment,  and  good  supervisory  relations. 

A  person’s  personality  and  motivation  in  the  work  place  is  a  “key  mediating 
mechanism”  (Hogan  1996;  Kanfer  and  Ackerman  2000;  Hogan  and  Holland  2003; 
Barrick  and  Mount  2005)  and  found  that  personality  can  influence  a  person’s  choice 
and  their  effort.  Schneider’s  (1987)  attraction-selection-attrition  (ASA)  model 
states  that  people  are  attracted  to  environments  that  are  similar  to  themselves. 
Extraverted  people  are  driven  to  places  where  people  are  animated  and  talkative. 
This  is  similar  to  research  in  a  learning  environment  that  finds  extraverted 
individuals  prefer  social  learning  environments,  such  as  a  group  training 
environment  or  classroom,  while  introverted  individuals  prefer  computer-based 
training  that  is  isolated  from  other  people  (Odum  and  Pourjalai  1994).  Research  has 
also  proved  this  concept  of  personality  influencing  the  environment  they  pick 
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(Barrick  et  al.  2003;  Judge  and  Cable  1997;  Schneider  et  al.  2001).  Barrick  et  al. 
(2003)  found  that  other  Big  5  dimensions  are  related  to  work  performance  and 
motivation.  Emotional  stability  is  connected  to  accomplishment  and  goal- 
orientation  based  on  high  and  low  emotional  stability. 

The  lure  of  rewards  surrounds  society,  from  credit  cards  promoting  their  card  and 
rewarding  with  points  to  use  later.  Banks  try  to  open  a  savings  account  for  free 
cash.  They  have  begun  tailoring  their  cards  and  rewards  to  individuals,  such  as  the 
“GE  Capital  Gains  Card”  that  rewards  for  saving  with  entertainment  and  travel  that 
is  given  if  a  person  conforms  to  the  behavior  for  the  card  (Walker  et  al.  2000). 
These  are  just  a  few  of  the  many  ways  that  businesses  and  the  workplace  are 
utilizing  reinforcers. 

A-8  Individual  Differences 

An  individual  difference  is  a  broad  term  that  can  refer  to  variables  such  as 
personality,  intelligence,  physiological  makeup,  and  prior  experiences.  The  Navy 
conducted  a  literature  review  on  individual  differences  that  have  an  impact  on 
training  outcomes.  The  differences  that  have  an  impact  on  outcomes  are  intrinsic 
motivation,  a  person’s  motivational  drive  to  learn,  goal  setting  capabilities;  a 
person’s  values  and  beliefs,  personality  differences,  organizational  commitments 
and  perceived  fairness,  and  attitude  on  training  all  were  found  to  have  an  impact  on 
training  outcomes  in  a  literature  review  (Schultz  et  al.  201 1).  This  section  attempts 
to  briefly  review  various  individual  differences  that  may  be  beneficial  for  inclusion 
in  the  MAT. 

A-8.1  Age 

The  generation  a  person  is  bom  in  can  affect  their  motivational  drive.  Matures  born 
in  1922  to  1945  value  conformity  and  sacrifice,  but  do  not  like  change  (Clare  2009). 
Baby  Boomers  from  1946  to  1963  value  personal  growth,  but  are  idealist  (Clare 
2009).  Generation  X  are  born  from  1964  to  1980s  are  more  literate  in  technology, 
but  are  also  more  independent  and  place  their  needs  in  front  of  their  jobs  (Clare 
2009).  Generation  X  also  sees  goal  orientation  as  a  primary  factor  in  job  satisfaction 
and  intention  to  remain,  but  not  so  for  Baby  Boomers  (Westerman  and  Yamamura 
2007).  The  Millennials  were  bom  in  the  early  1980s  to  2000  and  are  more  sociable, 
street  smart,  have  a  passion  for  learning,  but  expect  rewards  and  lack  loyalty  (Clare 
2009).  They  are  habituated  to  positive  praise  and  cannot  handle  negative  feedback 
efficiently  (Sujansky  and  Ferri-Reid  2010).  Simons  and  Simons  (2010)  also  found 
that  younger  workers  strongly  value  extrinsic  rewards,  but  also  are  not  as  willing 
to  work  longer  hours  to  achieve  the  reward.  The  younger  population  values  leisure 
and  time  away  from  work.  In  another  study  by  Ng  et  al.  (2010)  researched  on  23,413 
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millennial  undergraduate  students  in  Canada.  Millennial  rated  opportunities  for 
advancement  as  the  most  desirable  attribute,  second  was  working  with  and 
reporting  to  good  people,  third  was  training  opportunities,  and  fourth  was  work- 
life  balance.  Salary  fell  ninth  on  the  list  (Ng  et  al.  2010). 

A-8.2  Self-Efficacy 

The  attribution  theory  wants  to  know  why  events  occur.  Finding  out  a  person’s 
behavior  and  the  reason  for  the  behavior  connects  to  success  (Graham  and  Weiner 
1996).  When  a  person  is  successful  or  fails  they  look  for  a  reason  (Weiner  1986). 
The  person  could  believe  failure  is  from  environmental  (distractions,  teacher)  or 
personal  factors  (ability,  preparation).  Perception  of  failure  or  success  is  derived 
from  3  dimensions:  stability,  locus  of  control  (internal  or  external),  and 
controllability.  These  events  feed  into  a  person’s  self-efficacy  and  can  be 
influenced  by  feedback  (Lincht  1983;  Pintrich  and  Schunk  2002).  Identifying  the 
exact  cause  of  the  failure  or  success,  such  as  a  lack  of  effort  or  not  having  an 
appropriate  strategy,  should  be  considered  when  providing  feedback. 

A  person’s  self-efficacy  is  a  belief  about  a  person’s  capability  to  do  a  task  or  activity 
in  a  specific  context  or  domain  (Bandura  1997).  Self-efficacy  is  based  on  past 
successes  and  failures  on  specific  types  of  tasks.  A  person  with  high  self-efficacy 
tends  to  have  higher  levels  of  achievement  and  an  increase  in  persistence  on 
difficult  tasks  (Bandura  1997;  Pintrich  and  Schunk  2002).  Self-efficacy  is  also 
related  to  other  positive  outcomes  like  choice,  persistence,  cognitive  engagement, 
use  of  self-regulatory  strategies,  and  actual  achievement,  and  applies  to  all  ages, 
gender,  and  ethnic  groups  (Bandura  1997;  Pintrich  and  Schunk  2002).  It  is  fostered 
through  success  and  specific  praise  (not  meaningless).  Allowing  for  choice  on  the 
type  of  assessments  to  complete  can  enhance  self-efficacy  through  choice  based  on 
previous  successes.  Placing  self-efficacy  in  tutoring  systems  has  started  to  be 
explored  through  students  using  a  meter  about  their  ability  to  answer  the  problem 
successfully,  as  well  as  using  a  static  self-efficacy  model  from  a  pretest 
(McQuiggan  and  Lester  2006).  The  intelligent  tutor  can  assist  in  creating  positive 
affective  states  based  individual  motivators  that  effect  learning  such  as 
effectiveness,  transfer,  persistence,  and  increase  in  effort  (Zimmerman  2000) 
because  of  the  satisfaction  and  value  matches  the  individual’s  needs. 

The  regulatory  focus  theory  (Higgins  1998)  addresses  individual  self-regulatory 
differences.  Personality  plays  a  role  in  the  regulatory  focus  of  individuals  (Lanaj  et 
al.  2012).  Some  individuals  are  motivated  by  their  own  personal  hopes  and 
advancements  (promotion  focus),  and  thus  aim  at  positive  goals.  Others  are 
motivated  by  finishing  their  responsibilities  (prevention  focus).  People  who  fit  into 
a  prevention  focus  are  more  aware  of  risk,  safety,  and  avoid  errors  (Halvorson  and 
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Higgins  2013).  In  contrast  with  the  Big  5  (Digman  1990),  regulatory  is  closely 
linked  with  behavior.  Promotion  focus  is  linked  with  production  performance 
(Wallace  et  al.  2009)  and  sensitivity  to  the  presence  or  absence  of  rewards  (Kark 
and  Van  2007).  Another  meta- analysis  by  Condly  et  al.  (2003)  found  that  incentives 
increased  performance  by  22%  compared  with  companies  who  did  not  implement 
an  incentive  plan.  Promotion  focused  people  are  easier  motivated  by  individual 
incentive  schemes  and  prevention-focused  are  more  sensitive  to  group  incentives 
(Beersma  et  al.  2013). 

A-8.3  Personality 

The  interaction  of  personality  traits  and  learning  environments/strategies  impacts 
learning  outcomes  (Blickensderfer  et  al.  2003;  Komarraju  et  al.  2011). 
Additionally,  personality  traits  have  been  associated  with  cognitive,  affective,  and 
motivational  processes  (Matthews  and  Zeidner  2004).  The  Big  5  traits  have  been 
connected  to  behaviors,  academic  achievement,  and  job  performance  (Costa  and 
McCrae  1992;  Judge  et  al.  2007). 

The  Big  5  (Goldberg  1981)  is  one  of  the  more  commonly  used  personality  theories 
for  educational  psychology  (Dingman  1990;  Costa  and  McCrae  1986).  The 
following  are  the  Big  5  personality  traits: 

1.  Extraversion  or  introverted 

2.  Agreeableness 

3.  Conscientiousness 

4.  Neuroticism 

5.  Openness 

If  a  person  has  high  extraversion  they  are  more  social  and  talkative.  If  they  are  low 
extraversion  (e.g.,  introverted),  then  they  are  more  reserved.  Some  researchers  have 
associated  extraversion-introversion  with  the  “level  of  activity  in  the 
corticoreticular  loop,”  such  that  introverts  are  more  cortically  aroused  than 
extraverts”  (Eysenck  and  Eysenck  1985,  p.  197).  In  other  words,  introverts  have  a 
low  threshold  for  stimulation,  whereas  extraverts  have  a  high  threshold  for 
stimulation.  This  means  that  for  extraverts,  increased  arousal  results  in  increased 
performance,  while  with  introverts,  decreased  arousal  results  in  increased 
performance  (Eysenck  and  Eysenck  1985;  Matthews  and  Deary  1998). 
Consequently,  extraverts  are  considered  to  be  superior  to  introverts  in  performance 
in  high-arousing  environments,  as  well  as  view  social  activities  as  rewarding 
(Matthews  and  Zeidner  2004).  Introverts  are  seen  as  studying  and  a  positive  to 
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academic  achievement  (Poropat  2009).  Extraverts  are  seen  as  socializing  more  and 
not  studying  as  much  and  can  have  a  negative  result  on  academic  performance 
(Rolfhus  and  Ackerman  1999). 

People  who  are  more  agreeable  are  more  cooperative  and  those  who  are  less 
agreeable  are  competitive  and  sometimes  manipulative.  Agreeableness  is  seen  as  a 
positive  trait  for  academic  performance  (Lounsbury  et  al.  2003;  Farsides  and 
Woodfield  2003).  Agreeableness  is  seen  as  people  who  are  willing  to  put  their  own 
interest  after  others.  They  are  helpful  to  others,  have  self-control,  and  positive  on 
others  (Ahadi  and  Rothbart  1994;  Cumberland-Li  et  al.  2004).  They  have  been 
found  to  be  compliant,  soft-hearted,  and  trusting  (McCrae  and  Costa  1999). 

Conscientiousness  is  exhibited  by  a  person  who  pays  attention  to  details,  is 
organized,  and  is  goal  directed.  Conscientious  individuals  tend  to  be  self-focused 
and  self-governing  (Hmel  and  Pincus  2002).  They  are  also  responsible,  reliable, 
and  dependable  (Costa  and  McCrae  1992).  They  are  found  to  set  work  goals  for 
themselves  and  have  a  commitment  versus  individuals  who  scored  low  (Barrick  et 
al.  2003).  Conscientiousness  has  been  linked  to  GPA  (Conrad  2006;  Komarraju 
2011)  and  is  a  predictor  of  academic  success  (Blickenderfer  et  al.  2003;  Dollinger 
and  Orf  1991;  Furnham  et  al.  2003;  Paunonen  and  Ashton  2001).  A  person  that  is 
low  on  contentious  research  has  shown  that  they  are  more  likely  to  procrastinate  on 
a  task  (Steel  2007). 

Neuroticism  deals  with  emotions.  Individuals  with  high  neuroticism  have  a  greater 
level  of  emotions  such  as  stress,  anxiety,  irritability,  or  sadness.  If  one  is  low  on 
neuroticism,  then  they  are  more  stable  with  their  emotions  (McCrae  and  John 
1992).  Some  researchers  attribute  this  to  “individual  differences  in  excitability  and 
emotional  responsiveness,  which  are  reflected  in  autonomic  activation”  (Eysenck 
and  Eysenck,  p.  232).  Neuroticism  has  been  characterized  by  working  memory 
deficit  and  pessimistic  outlook  leading  to  negative  reactions  to  demanding 
environments,  emotion  focus  rather  than  task-focus,  and  low  self-esteem 
(Matthews  1999).  Across  all  types  of  neuroticism,  coping  strategies  appear  to  be 
related  to  the  particular  environment  in  which  the  individual  is  operating,  which 
suggests  that  the  MAT  should  assess  an  individual’s  comfort  in  a  given  situation  or 
environment.  This  is  consistent  with  Matthews  and  Zeidler’s  (2004)  view  of 
personality  having  both  fixed  traits  as  well  as  dynamic  states,  which  are  dependent 
on  the  specific  environment.  High  neuroticism  is  deemed  hurtful  in  academic 
performance  (Chamorro-Premuzic  and  Fumham  2003).  Introjected  motivation  was 
positively  related  to  neuroticism  when  looking  at  personality  and  motivation 
relationship  (Phillits  et  al.  2003).  Emotional  stability  has  been  linked  with  self- 
efficacy  motivation  (Jedge  et  al.  1998) 
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Openness  allows  a  person  to  have  more  interest  and  creativity.  Individuals  low  on 
openness  are  traditional  and  may  not  like  change  or  abstract  thinking.  Openness  is 
correlated  positively  with  academic  performance  (Lounsbury  et  al.  2003;  Farsides 
and  Woodfield  2003)  and  intellectual  curiosity  (Komarraju  et  al.  2011). 

Komarraju  and  Karau  (2009)  looked  at  the  links  between  Big  5  traits,  academic 
motivation,  and  academic  achievement.  The  results  showed  that  conscientiousness 
was  the  most  influential  and  had  relationships  with  all  the  variables:  intrinsic, 
extrinsic,  amotivation,  and  GPA.  They  had  higher  scores  on  both  intrinsic  and 
extrinsic  motivation  had  the  lowest  amotivation  and  the  highest  achievement. 
Agreeableness  and  openness  were  more  connected  with  achievement.  People  who 
had  higher  intrinsic  motivation  also  had  higher  tendencies  toward  openness. 
Neurotic  individuals  were  seen  to  have  a  higher  amotivation.  The  results  showed 
that  personality  traits  effect  motivation  and  achievement.  It  also  suggested  that 
conscientious  students  who  are  organized  and  disciplined  are  more  motivated. 
Extraversion  was  related  with  extrinsic  motivation  and  conscientiousness  was  not 
associated  with  amotivation.  Similar  results  were  obtained  in  related  study  in  201 1 
(Komarraju  et  al.  2011).  In  summary,  there  is  sufficient  evidence  that  there  is  an 
interaction  with  an  individual’s  personality  traits,  preference  to  a  given 
environment  (e.g.,  situational  context)  and  motivation.  Specifically,  this  interaction 
demonstrates  how  an  individual’s  personality  traits  may  predispose  them  to  view 
certain  types  of  tasks  as  rewarding — or  punishing.  If  the  MAT  is  able  to  determine 
both  an  individual’s  personality  traits  and  preference  for  the  specific  learning 
task/environment,  this  information  can  be  used  to  recommend  a  motivator  that  will 
be  viewed  as  a  reward. 

A-8.4  Value 

A  person’s  values  are  relatively  stable  over  time.  They  are  what  a  person  finds 
important  and  it  is  a  motivation  to  a  person.  It  is  in  essence  a  long-term  goal  that 
stands  in  different  context  and  points  of  time.  (Schwartz  1997;  Schwartz  and  Bilsky 
1990).  Schwartz  value  theory  has  10  basic  values  for  humans:  power  (social  status, 
control,  dominance),  achievement  (successful,  capable,  ambitious,  influential), 
hedonism  (pleasure  and  enjoyment  for  yourself),  stimulation  (excitement,  novelty, 
challenge),  self-direction  (creativity,  freedom,  independent,  choosing  own  goal), 
universalism  (wisdom,  equality,  peace,  protecting),  benevolence  (helpful,  honest, 
forgiving,  loyal,  responsible),  tradition  (religion,  tradition,  customs),  conformity 
(politeness,  obedient,  self-discipline,  honoring  elders  or  parents),  and  security 
(family  and  national  safety,  clean,  orderly).  Schwartz  made  broader  values 
openness  to  change,  self-enhancement,  self-transcendence,  and  hedonism.  This  has 
been  linked  to  personality  traits.  A  person’s  values  and  traits  are  linked  by  our 
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temperament  we  are  born  with  and  so  value  may  affect  traits  and  traits  may  affect 
values.  Extraversion  correlates  positively  with  stimulation,  achievement,  and 
hedonism  and  negatively  to  tradition  values  (Roccas  et  al.  2002). 

A-8.5  Individual  Differences  Rewards  Used  in  Learning  Games 

In  video  games,  which  is  perceived  to  be  intrinsically  motivating,  creators  also  a 
need  to  incorporate  extrinsic  rewards  for  individual  differences  such  as  a  task  that 
might  be  seen  as  boring  (Schneider  1987).  Rewards  used  in  games  are  thought  to 
be  intrinsically  motivating  because  they  are  fun  and  enjoyable  (Robson  et  al.  2015). 
Some  motivational  strategies  used  in  games  (Hamari  et  al.  2014)  are  points, 
leaderboards,  achievements/badges,  levels,  story/theme,  clear  goals,  feedback, 
rewards,  progress,  and  challenge.  Evidences  shows  that  rewards  used  in  games  are 
motivating  in  game-based  learning  (Jabbar  and  Felicia  2015). 

In  a  search  on  all  reward  systems  used  online  and  on  apps  based  on  research,  Lewis 
et  al.  (2016)  found  that  the  most  widely  used  reward  system  was  with  points 
(Caffazzo  et  al.  2012;  Agoristas  et  al.  2014;  Allam  et  al.  2015;  Boendermaker  et  al. 
2015).  Points  were  used  on  55%-73%  of  apps  on  Apple  (Lister  et  al.  2014;  Payne 
et  al.  2015).  The  next  most  popular  were  the  use  of  achievements/badges/medals 
that  are  given  when  the  behavior  is  changed  or  meeting  a  level.  Agoritsas  et  al. 
2015;  Allam  et  al.  2015).  The  next  popular  were  tangible  rewards  where  people 
receive  credit  to  the  iTunes  store  stickers,  ribbons,  or  medals  on  a  game  board  and 
other  tangible  rewards.  Other  rewards  that  were  popular  were  currency,  likes  by 
others  (on  social  media  posts),  animated  feedback,  and  kudos  (virtual  gifts  that 
could  be  shared).  However,  none  of  the  studies  reported  on  the  effects  of  the 
rewards  on  desired  outcomes.  Lewis  et  al.  (2016)  placed  them  with  taxonomy: 
points,  achievement/badges/medals  and  animated  rewards  were  for  glory  (Hallford 
and  Hallford  2001).  Kudos  and  likes  were  social  rewards  (Rigby  2009),  currency 
like  power-ups  (Kamel  et  al.  2015)  or  game  progression  (Jones  et  al.  2014). 

Research  on  personality  differences  and  video  games  found  that  extraversion  is 
connected  to  relationship,  adventure,  and  achievement  motivation,  agreeableness 
to  escapism,  achievement  motivation  (Park  et  al.  201 1),  and  competence  motivation 
(John  and  Srivastava  1999).  Openness  has  been  linked  to  motivation  for  autonomy 
(Johnson  and  Gardner  2010).  Motivation  impacts  intrinsic  motivation  that  makes  it 
enjoyable  and  could  change  the  reward  selection  based  on  differences.  Personality 
traits  are  also  connected  with  reward  preferences.  Conscientiousness  and  openness 
have  been  associated  with  greater  preference  for  rewards  (Karanam  et  al.  2010)  and 
extraversion  is  motivated  to  achieve  rewards  in  goal-directed  activities.  Unfulfilled 
needs  is  associated  with  preference  of  rewards  which  is  linked  to  neuroticism, 
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conscientiousness  and  agreeableness.  A  person  who  is  easily  bored  needs  more 
short-term  goals  (Staewen  et  al.  2014). 

Research  has  been  started  on  individual  differences  and  reward  selection.  Some 
motivational  strategies  used  in  games  (Hamari  et  al.  2014)  are  points,  leaderboards, 
achievements/badges,  levels,  story/theme,  clear  goals,  feedback,  rewards,  progress, 
and  challenge.  Karanam  and  colleagues  (2010)  found  that  participants  who  had 
high  conscientiousness  and  openness  preferred  rewards,  participants  who  were  high 
on  extraversion  and  openness  liked  challenges  and  missions.  These  anecdotes  are 
consistent  with  the  description  of  these  personality  traits  applied  to  learning  that 
was  previously  discussed. 

Nagle  et  al.  (2016)  used  performance-contingent  rewards  (round  in  a  certain  time 
frame),  task-contingent  rewards  (completing  a  round),  and  no  rewards.  The  awards 
were  given  with  points  and  animation.  The  Big  5  Personality  traits  were  assessed  a 
priori  with  a  questionnaire  and  their  intrinsic  motivation  was  measured  by  their  task 
performance.  The  study  showed  that  openness  and  extraversion  predicted  intrinsic 
motivation  and  performance  in  performance-contingent  rewards.  Extraverted 
individuals  found  rewarded  based  on  performance  appealing  because  they  are 
achieving.  They  also  found  that  openness  has  been  related  to  wanting  autonomy  in 
games  (Johnson  and  Gardner).  Neuroticism  is  liked  to  short-term  rewards 
(Manning  et  al.  2014)  and  was  seen  as  negative  for  task-contingent  rewards  since 
they  didn’t  need  to  perform  to  be  awarded  often.  Effort  was  seen  through 
conscientiousness  for  both  kinds  of  task  (performance-task).  Extraversion  wasn’t 
effected  by  rewards.  This  study  showed  that  reward  selection  is  different  based  on 
different  combination  of  personality  traits. 

A-8.6  Summary 

Different  people  place  different  values  on  rewards  and  it  is  important  to  find 
individual  differences  to  place  a  perfect  match.  Personality  is  an  individual 
differences  variable  that  affects  whether  they  perceive  an  activity  as  a  motivator. 
There  are  different  ways  a  person  can  measure  personality  such  as  the  Big  5  model 
of  personality  (Wiggins  and  Trapnell  1996).  The  Big  5  is  one  of  the  most  common 
models  to  deal  with  personality  that  has  been  used  in  research  pertaining  to 
motivation  and  learning  and  is  measured  by  questionnaires.  With  respect  to  the 
MAT,  this  research  suggests  the  possibility  of  creating  a  set  of  reward 
recommendations  that  are  linked  with  personality  traits.  For  example,  if  someone 
is  extraverted,  a  reward  may  be  the  opportunity  to  use  social  media  in  conjunction 
with  their  learning  activity. 
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In  summary,  to  better  understand  an  individual’s  needs  with  respect  to  increasing 
their  motivation  requires  an  understanding  of  their  personality  traits.  This  will  aid 
in  projecting  how  a  specific  motivational  strategy  may  be  perceived  and  whether  it 
will  be  useful. 

A-9  Physiological  Measures 

Physiological  and  behavioral  indicators  of  motivation  measure  objectively  during 
and  instructional  session  to  identify  changes  in  the  students’  affect  that  can  reflect 
their  motivational  state.  These  types  of  measures  can  provide  objective  insight  into 
how  a  student  is  responding  to  a  specific  motivational  strategy  without  potential 
effect  of  self-report  tending  to  reflect  a  more  positive  response  than  actually 
experienced  (Woolf  et  al.  2009).  The  use  of  these  types  of  measures  to  make 
inferences  regarding  the  effectiveness  of  a  reinforcer  requires  comparing  their 
responses  while  receiving  the  reinforcement  to  a  baseline  measure  taken  prior  to 
the  start  of  the  instructional  session. 

Specific  physiological  measures  that  can  provide  insight  on  motivational  state 
include  skin  conductance,  heart  rate  (including  cardiac  variability  and  interbeat 
interval  [IB  I]),  electromyogram  [EMG],  and  muscle  tension),  and 
electroencephalogram  (EEG;  brain  activity).  Skin  conductance,  heart  rate  (HR), 
and  skin  temperature  evaluate  autonomic  nervous  system  activity,  specifically  the 
sympathetic  (SNS;  controls  activity  when  energy  is  expended)  and  parasympathetic 
(PNS;  controls  activity  during  periods  of  rest)  nervous  systems  (Andreassi  1995). 
EMG  provides  a  measure  of  somatic  nervous  system  activity,  and  EEG  provides  a 
measure  of  CNS  activity.  Table  1  summarizes  the  information  that  can  be  gleaned 
from  real-time  physiological  measures  during  an  instructional  session. 


Table  A-l  Physiological  measure  overview 


Physiological  measure 

System 

Affect  type 

Skin  conductance 

SNS 

Arousal,  engagement,  boredom 

Heart  rate 

SNS 

PNS 

Arousal,  engagement,  fear 

Stress,  frustration,  anxiety 

EMG 

Somatic 

Stress,  frustration 

EEG 

CNS 

Workload,  engagement 

Evaluation  of  changes  in  HR  can  be  used  to  evaluate  changes  in  arousal,  with 
increased  HR  indicated  increase  arousal  due  both  positive  (e.g.,  engagement)  and 
negative  (e.g.,  fear)  emotion,  as  well  as  distress  with  a  sudden  decrease  in  HR 
related  to  emotions  such  as  sadness  or  disgust.  It  has  also  been  recommended  that 
HR  increases  are  more  reflective  of  mental  workload  (Grossman  1992),  and 
therefore,  could  indicate  that  the  student  is  extending  effort  or  engaged  in  their 
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learning  task.  HR  variability  describes  the  variability  in  the  interbeat  interval,  or 
periods  between  heart  beats,  and  has  been  used  to  evaluate  changes  in  emotional 
responding  (Appelhans  and  Luecken  2006).  HR  data  are  typically  collected  using 
photoplethysmography  (measure  of  blood  flow  volume)  or  electrocardiogram 
(ECG).  Similar  to  skin  conductance,  HR  data  can  be  obtained  through  minimally 
invasive  sensors,  which  use  photoplethysmography.  Personality  differences  should 
also  be  considered  in  evaluating  HR  responses.  For  example,  individuals  high  in 
neuroticism,  who  have  a  tendency  to  negatively  interpret  the  environment,  may 
exhibit  greater  changes  to  the  presentation  of  a  reinforcer  or  feedback  than  those 
lower  in  neuroticism  (Fichera  and  Andreassi  2000). 

It  has  been  demonstrated  the  increased  EEG  activity  is  associated  with  increases  in 
arousal  (Bradley  and  Lang  2000).  EEG  has  also  been  used  to  evaluate  student 
workload  and  engagement  (Berka  et  al.  2007).  While  EEG  devices  may  be 
undesirable  due  to  the  requirement  for  wearing  sensors  attached  to  one’s  head,  the 
technology  is  progressing  so  that  it  is  more  user  accepted.  Additionally,  when 
thinking  of  learning  in  a  virtual  reality  (VR)  or  augmented  reality  (AR)  enabled 
environment,  the  integration  of  EEG  with  an  AR/VR  headset  may  be  a  more 
realistic  solution. 

Behavioral  measures  can  also  provide  another  source  of  real-time  assessment  of 
change  in  affect  in  response  to  triggers  such  as  the  provision  of  a  reinforcer.  Eye 
blink  rate  (EBR)  has  been  associated  with  dopamine  DA  release,  with  increased 
eye  blinks  associated  with  increased  secretion  of  DA  and  a  decrease  in  eyeblink 
activity  associated  with  decreased  DA  secretion  (Slagter  et  al.  2015).  As  discussed 
in  the  prior  section,  increases  in  DA  secretion  have  been  associated  with  positive 
rewards,  whereas  decreases  in  DA  occur  in  response  to  negative  rewards/ 
punishment  (Daw  and  Shohamy  2008).  Pupil  dilation  and  eye  tracking  (Conati  et 
al.  2013)  have  been  used  to  evaluate  student  engagement  and  recommended  for 
adaptive  learning  environments.  Finally,  other  behavioral  measures  of  motivation 
include  speed  or  response  rate  to  answering  questions  or  performing  a  task,  in 
which  increased  speed  indicates  increased  motivation  (when  the  task  is  performed 
well),  and  performance  of  the  task  itself  (Toure-Tillery  and  Fishbach  2014). 

A- 10  Assessments  Classifications 

We  have  discussed  how  a  reinforcer  can  increase  motivation,  visual  attention,  and 
retention.  Now  let’s  focus  on  how  to  measure  current  level  and  changes  in  student 
motivation  and  determine  how  an  individual’s  personality  profile. 
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A-10.1  Motivation  Rewards 

In  the  review  of  literature,  motivation  is  primarily  measured  the  following  ways: 
1)  cognitive  measures  (recall),  2)  subjective  self-reports  that  ask  how  satisfied  the 
individual  feels,  3)  behavioral/performance  assessment,  and  4)  physiological 
measures  (Toure-Tillery  and  Fishbach  2014). 

Another  means  of  assessing  motivation  is  to  evaluate  an  individual’s  progress  in 
meeting  goals  that  they  have  set  or  have  been  set  by  an  external  system.  Goals  can 
be  assessed  by  evaluating  an  increase  in  time  on  a  task,  which  can  indicate  that  the 
individuals  are  focused  and  more  conscientious  about  getting  the  answer  correct 
with  a  deeper  level  of  learning.  Conversely,  a  decrease  in  time  to  complete  can 
indicate  a  decreased  level  of  motivation  due  to  a  lower  level  of  engagement 
resulting  in  the  amount  of  time  and  effort  expended  on  a  task.  However,  it’s 
important  to  determine  the  context  of  a  situation.  For  example,  if  there’s  a  time 
constraint  for  completing  a  task,  a  decrease  in  time  could  be  an  indication  of  high 
motivation  if  the  individual  is  focusing  on  achieving  the  time  sensitivity  goal 
associated  with  the  task.  The  Patterns  of  Adaptive  Learning  Survey  (PALS)  has 
been  used  in  a  variety  of  cultures  and  ages  (Midgley  et  al.  1998)  to  assess 
achievement  of  goals  and  the  individual’s  type  of  orientation  toward  the  goal. 

There  are  assessments  for  both  intrinsic  and  extrinsic  motivation,  which  describe 
how  a  person  is  motivated  based  on  the  context  or  particular  topic  or  even  in 
general.  One  example  is  the  Harter  (1981)  self-report  scale  of  intrinsic  versus 
extrinsic  motivation.  This  scale  measures  intrinsic  versus  extrinsic  motivation  in 
the  classroom,  with  3  scales  assessing  motivation  (preference  for  challenge, 
curiosity/interest,  mastery)  and  2  scales  assessing  cognitive-informational 
structures  (such  as  independent,  judgment,  internal  criteria).  Another  examples  are 
the  Intrinsic  Motivation  Inventory  (IMI;  Ryan,  2007)  used  to  evaluate  intrinsic 
motivation  and  self-regulation.  The  IMI  comprises  45  items  that  make  up  7 
subscales,  and  it  is  administered  post-task.  Barron  and  Harackiewicz  (2001)  have 
investigated  intrinsic  motivation  with  questions  regarding  items  that  the  individual 
likes  or  enjoys.  Situational  interest,  which  considers  the  context  of  the  task,  has  also 
been  considered  (Harackiewicz  et  al.  2000). 

The  AMS-C  28  (Vallerand  et  al.  1992)  is  a  28-item  measure  of  academic  motivation 
used  to  determine  reasons  why  students  attended  college.  Its  7-factor  structure  is 
based  on  Deci  and  Ryan's  (1985)  self-determination  theory.  The  7  subscales 
comprise  a)  3  measures  of  intrinsic  motivation:  intrinsic  motivation  to  know  (IM  to 
know),  intrinsic  motivation  toward  accomplishments  (IM  to  accomplish  things), 
and  intrinsic  motivation  to  experience  stimulation  (IM  to  experience  stimulation); 
b)  3  measures  of  extrinsic  motivation:  identified  regulation,  introjected  regulation, 
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and  external  regulation;  and  c)  amotivation.  Although  each  subscale  is  measured 
by  only  4  hems,  the  reliability  and  validity  of  the  AMS-C  28  has  been  established 
with  measures  of  internal  consistency,  test-retest  reliability,  concurrent  validity, 
and  construct  validity  (Vallerand  et  al.  1992). 

The  Instructional  Materials  Motivation  Survey  (IMMS)  is  a  self-report  based 
assessment  to  for  evaluating  how  students  view  the  instructional  content  itself  as 
motivational  or  amotivational  based  on  the  ARCS  model  of  motivation  (Loorbach 
et  al.  2015).  The  ARCS  model  (Keller  2010)  as  described  previously  assumes  that 
motivation  is  dependent  upon  4  interrelated  constructs:  attention,  relevance, 
confidence  and  satisfaction.  The  IMMS  is  a  36-item  survey  used  to  assess  the 
effects  of  instruction  on  a  student’s  motivation.  The  IMMS — as  well  as  ARCS — 
was  developed  with  a  traditional  classroom  based  instructional  setting  in  mind,  with 
most  validation  studies  of  the  IMMS  being  completed  in  a  classroom  setting. 
Additional  researchers  have  evaluated  its  use  in  the  context  of  self-directed, 
technology-based  learning,  such  as  validation  of  the  IMMS  in  a  self-directed 
instructional  setting  aimed  at  working  with  technology  (Loorbach  et  al.  2015; 
(Huang  et  al.  2010;  Huang  and  Hew  2016).  The  results  of  Loorbach  et  al.’s 
validation  of  the  IMMS,  and  the  Reduced  IMMS  (RIMMS),  suggest  that  the  12- 
item  RIMMS  more  accurately  assesses  student  motivation  in  a  self-regulated, 
technological  learning  environment.  Huang  and  Hew  implemented  an  online 
version  of  the  IMMS  to  evaluate  learner  motivation  levels  during  enrollment  in  a 
massive  open  online  course  (MOOC).  Another  self-report  questionnaire  is  the 
online-motivation  questionnaire  (OMQ),  which  specifically  evaluates  amount  of 
effort  a  student  plans  to  expend  on  a  task  (Trimmers  et  al.  2013).  The  authors  used 
3  of  the  7  dimensions  the  OMQ  assesses  (task-value  beliefs,  success  expectancy 
and  effort)  to  evaluate  the  effect  of  instructional  feedback  and  students  interest  in 
the  feedback  on  their  motivation  and  learning  outcomes. 

The  input-environment-outcome  (I-E-O)  model  is  an  analysis  method  for  assesses 
how  student  internal  characteristics  (inputs)  and  external  (environment)  variables 
impact  a  student’s  learning  (House  1999) — with  internal  characteristics  including 
intrinsic  motivation  and  external  variables  including  rewards  or  other  motivators. 
House  used  the  I-E-0  assessment  model  to  evaluate  the  interaction  of  student 
internal  characteristics  (high  school  GPA,  self-rated  academic  ability  and 
expectations  of  graduating  with  honor)  and  learning  environment  variables 
(hours/week  study  time,  participation  in  a  group  project,  work  outside  of  college 
and  commute  time)  on  learning  outcomes  (student  satisfaction  with  their  college 
and  completion  of  bachelor’s  degree).  The  analysis  method  comprises  3  steps: 
1)  correlations  between  the  input  and  environment  variables,  2)  correlations 
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between  input/output  and  outcome  measures,  and  3)  Causal  Analytical  Modeling 
via  Blocked  Regression  Analysis  (CAMBRA)  to  evaluate  the  overall  model. 

Self-efficacy  is  assessed  through  questionnaires.  The  MSLQ  (Pintrich  et  al.  1993) 
and  PALS  (Midgley  et  al.  1998)  include  measures  to  assess  self-efficacy  through 
questions  that  the  that  the  individual  rates  on  a  5-  to  7-point  Likert  scale. 

A-10.2  Grit 

Grit  is  a  person’s  ability  to  persist  at  a  lifetime  or  long-term  goal  either  professional 
or  educational  despite  failures  and  challenges  (Duckworkth  and  Quinn  2009; 
Vallerand  et  al.  2014).  A  person  with  higher  grit  is  more  effective  (Robertson-Kraft 
and  Duckworth  2014).  They  tend  to  practice  more  because  of  their  intrinsic  passion 
and  it  shows  in  their  level  of  skill  and  performance  (Vallerand  et  al.  2014).  It  is 
assessed  using  self-reports  or  questionnaires  such  as  the  Brief  Grit  Scale 
(Duckworth  and  Quinn  2009).  The  Grit  Gauge  (Stoltz  et  al.  2016)  assesses  a 
person’s  growth,  resilience,  instinct,  and  tenacity  and  robustness.  A  person’s 
growth  is  a  person  who  seeks  for  new  ideas/ways,  resilience  is  how  one  deals  with 
challenges,  instinct  is  finding  one’s  path  to  the  goal  in  the  best/right  way,  and 
tenacity  is  one’s  persistence.  The  robustness  is  how  a  person  deals  with  challenges 
over  time  (Stoltz  et  al.  2016).  It  is  closely  related  to  the  personality  trait  of 
conscientiousness  and  related  to  academic  achievement  (Duckworth  and  Quinn 
2009).  Grit  is  seen  as  a  part  of  conscientiousness  because  the  person  is  self-dutiful 
and  achievement  oriented.  (Duckworth  et  al.  2007). 

Schools  have  reinforcement  assessments  used  to  try  and  find  the  perfect  match  for 
a  reinforcer.  One  that  is  widely  used  is  the  Dunn-Rankin  Reward  Preference 
Inventory  (DRRPI;  Cartwright  and  Cartwright  1970).  The  DRRPI  classifies  the 
rewards  into  adult  approval,  peer  approval,  tangible  rewards,  competition,  and 
independence.  Cartwright  and  Cartwright  (1970)  report  of  a  study  demonstrated 
that  the  DRRPI  was  a  valid  indicator  of  reward  preferences  for  the  primary-age 
learning  disabled  students  but  had  limited  uses  for  all  students  in  the  classroom.  It 
is  used  in  schools,  but  there  is  very  little  to  no  literature  on  the  DRRPI.  Similar  to 
the  DRRPI,  the  forced-choice  survey  asks  students  2  sentences  and  they  have  to 
pick  the  statement  that  identifies  the  reward  they  would  prefer  (Gable  1991). 
Schools  typically  used  the  forced-choice  survey  in  conjunction  with  the  Positive 
Behavioral  Intervention  Schools  survey.  This  latter  instrument  classifies  motivation 
based  on  a  measure  of  adult  approval,  competitive  approval,  peer  approval, 
independent  rewards  and  consumable  rewards.  There  is  no  literature  about  the 
validity  of  the  test  but  it  is  used  in  schools. 
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Other  inventories  such  as  the  Behavioral  Assessment  Guide  (ABA-Instituut  1993) 
is  used  to  determine  both  children  and  adult  reinforcement  preferences.  It  is  rated 
on  a  scale  from  not  at  all  to  very  much.  The  children’s  section  is  broken  into  food 
items,  toys  and  playthings,  entertainment,  sports  and  games,  music,  arts,  crafts, 
excursions,  and  community  events  that  could  be  reinforcing.  It  also  evaluates  how 
social  interaction,  academic/classroom,  domestic  activities,  personal  appearance, 
token  reinforcers,  and  other  possible  reinforcers  are  perceived  by  the  student.  Then, 
the  student  list  activities  they  do  multiple  times  a  days  and  how  much  time.  For 
adults,  the  Behavioral  Assessment  Guide  examines  different  variables  under 
entertainment,  hobbies,  food,  beverages,  sports,  excursions,  social,  helping,  books 
and  magazines,  and  personal  appearance.  Then  it  asks  how  long  a  person  likes  to 
do  things  like  watch  TV,  sleep,  read,  be  alone,  or  with  others.  This  information  is 
used  to  determine  the  types  of  rewards  likely  to  encourage  motivation. 

A-10.3  Personality 

With  respect  to  assessing  the  Big  5  personality  traits,  the  10  Item  Personality 
measure  (TIPI)  is  a  short  form  assessment  of  personality  with  only  2  items  per  each 
Big  5  trait  (Gosling  et  al.  2003).  Another  instrument  for  assessing  the  Big  5 
personality  traits  is  the  International  Personality  Item  Pool  (IPIP;  Goldberg  2016). 
There  is  a  more  extended  version  that  considers  more  traits  than  the  Big  5  such  as 
humor  and  toughness,  called  the  AB5C  Model  of  Personality.  Other  inventories 
have  been  the  NEO  5  Factor  Inventory  (NEO-FFI:  Costa  and  McCrae  1992)  that 
uses  a  5-point  scale  on  the  5  factors  there  is  also  the  Eysenck  Personality 
Questionnaire-Revised  Short  Form  (EPQ-R  Short  Form;  Eysenck,  Eysenck,  and 
Barrett  1985). 

A-10.4  Games,  Motivation,  and  Assessment 

Other  classifications  from  games  we  know  that  players  need  rewards  for 
advancement  cooperation/compete,  review,  sociality.  Players  like  to  review  their 
rewards  and  they  like  to  share  about  their  rewards  (Wang  and  Sun  2011).  Others 
games  classify  by  relationship,  adventure,  and  achievement  motivation, 
agreeableness  to  escapism  and  achievement  motivation  (Park  et  al.  2011)  and 
competence  motivation  (John  and  Srivastava  1999).  Games  have  been  used  for  a 
variety  of  applications  from  training,  learning,  and  rehabilitation.  Rewards  in 
games  can  be  long  and  short  term,  and  are  administered  as  a  high  score,  animations, 
feedback  messages,  and  so  on.  These  are  based  on  different  characteristics  of 
people  or  player  type.  (Staewen  et  al.  2014).  Rewards  and  goals  are  important  in 
computer  games  (Salen  and  Zimmerman  2004).  Bartle  (2003)  found  that  players 
have  different  styles  and  make  different  choices  for  different  scenarios.  He 
classified  players  into  groups  based  on  how  they  played.  These  classifications 

Approved  for  public  release;  distribution  is  unlimited. 

88 


included  explorers,  achievers,  killers,  and  socializers  (Bartle  2003)  and  then  Yee 
(2002)  added  achievement,  social  interaction,  and  immersion.  For  the  games  to 
apply  to  everyone,  they  have  to  create  with  a  range  of  rewards  when  playing. 
Kuhlman  and  Marshello  (1975)  classified  players  by  cooperative,  competitive,  or 
individualistic.  Bateman  and  Boon  (2006)  found  4  clusters  of  players  styles: 
conquerors  (high  on  thinking),  managers,  wanderers  (high  on  feeling  and 
perceiving),  and  participants.  They  linked  the  players  to  personality  traits  drawn 
from  the  Myer-Briggs  Type  Indicator.  Based  on  internal  and  external  factors, 
Przybylski  and  colleagues  developed  Player  Experience  of  Need  Satisfaction 
(PENS)  to  assess  players  for  competence,  autonomy,  relatedness,  intuitive  controls, 
and  presence/immersion.  PENS  is  a  21 -item  survey  using  a  7-point  scale  and  5 
subscales. 

A-ll  Summary 

There  are  a  wide  range  of  questionnaire-based  assessments  that  have  been  applied 
in  a  wide  range  of  educational  setting  to  a  variety  of  student  populations.  Common 
across  these  various  tools  is  the  assessment  of  the  student’s  preference  or  comfort 
within  a  specific  learning  domain,  task  or  environment.  Similarly,  questionnaires 
are  the  primary  form  of  personality  trait  assessment. 

A-12  Conclusion 

The  literature  discussed  in  this  report  provides  evidence  that  both  intrinsic  and 
extrinsic  motivation  impact  the  overall  effectiveness  of  an  instructional  or 
otherwise  educational  activity.  Evidence  was  presented  from  behavioral,  cognitive, 
and  physiological/neurological  studies.  Additionally,  the  research  discussed 
recommends  that  the  context  of  the  subject  or  task  to  be  learned  and  the  student’s 
preference  or  comfort  within  that  context  influences  their  motivation.  The  context 
also  affects  the  types  of  rewards  that  may  motivate,  or  amotivate,  a  student. 

Research  was  also  presented  on  the  role  of  personality  traits  on  how  an  individual 
perceives  their  environment,  and  how  they  respond  to  various  feedback  and  reward 
styles.  There  was  also  discussion  on  how  difference  environments  are  more  or  less 
suited  to  different  personality  profiles.  Consequently,  there  is  evidence  that 
suggests  the  use  of  an  individual’s  personality  profile  along  with  their  known 
preference  or  comfort  for  a  given  task,  can  be  used  to  tailor  the  type  of  reward  or 
reinforcer  used  to  motivate  the  student. 

The  reinforcer  can  be  seen  as  an  instructional  strategy  that  is  based  on  the  learners 
motivation.  Learner  motivation  is  one  of  the  classifications  important  for  learner 
characteristics  (Goldberg  et  al.  2012).  Research  has  shown  improvement  in  the 
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learning  rate  and  retention  when  using  reinforcers  (rewards).  Customizing  for  an 
increase  in  motivational  level  based  on  individual  differences,  when  properly 
implemented  with  consideration  of  the  aforementioned,  has  the  potential  to 
optimize  learning  outcomes  when  using  GIFT. 

However,  there  is  much  overlap  in  the  construct  of  motivation  with  other  student 
affective  characteristics.  Self-efficacy,  which  is  an  individual’s  belief  that  they  can 
successfully  complete  a  task  or  accomplish  a  goal  (Bandura  1997),  weighs  in  on  an 
individual’s  intrinsic  motivation,  or  interest  in  performing  a  task.  The  study  of 
student  autonomy,  which  is  the  individual’s  perceived  degree  of  control  over  their 
success  in  a  learning  environment  (Noels  et  al.  1999),  can  be  viewed  as  type  of 
motivational  strategy.  Specifically,  some  students  demonstrate  increased 
motivation  when  they  have  more  control  over  their  learning  progress  (Kember  et 
al.  1999).  The  point  of  this  discussion  is  that  attempting  to  precisely  measure 
motivation  may  be  a  challenging  endeavor,  and  is  likely  intermingled  with  some  of 
these  related  variables.  A  recommendation  is  to  leverage  the  prior  research 
completed  on  individual  characteristics  and  the  types  of  interactions  that  they  enjoy 
or  perceive  as  rewarding  for  developing  a  framework  for  implementing  a 
motivational  strategy.  Specifically,  an  evaluation  of  personality  up  front  in  the 
learning  process  may  provide  an  entry  point  for  identifying  the  types  of 
instructional  strategies  or  interventions  that  may  increase  student  motivation.  The 
effectiveness  of  these  strategies  can  be  evaluated  through  the  student’s  learning 
performance,  level  of  effort  and  time  spent  on  task  in  both  real-time  and  after  each 
instructional  session.  These  evaluations  can  be  used  to  update  the  value  of 
effectiveness  of  the  strategy  in  the  long-term  learner  mode. 
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B-l  Demographics 

These  are  obtained  through  demographic  questions  (age,  gender,  etc.). 


B-2  Personality 

One  of  the  most  commonly  used  set  of  personality  traits  is  the  Big  5  (Goldberg 
1981;  Dingman  1990;  Costa  and  McCrae;  1986)  (Table  B-l). 

Table  B-l  Big  5  personality  traits 


Trait 

Definition 

Facets  of  the  trait 
(Johnson  2014) 

Openness 

Openness  is  a  person  that  has  traits  of  being 
independent,  imaginative,  many  interests, 
and  interest  to  new  experiences. 

Imagination 

Artistic  Interest 

Emotionality 

Adventurousness 

Intellect 

Liberalism 

Conscientiousness 

The  trait  of  being  thoughtful,  detailed, 
organized,  self-disciplined,  goal-directed 
and  responsible. 

Self-efficacy 

Orderliness 

Dutifulness 

Achievement-striving 

Self-discipline 

Cautiousness 

Extraversion 

This  trait  is  related  to  an  individual  that  is 
social,  talkative,  active,  and  positive 
emotion 

Friendliness 

Gregariousness 

Assertiveness 

Activity  level 

Excitement  seeking 

Cheerfulness 

Agreeableness 

Agreeableness  is  a  person  that  is  good- 
natured,  thinks  of  others,  helpful,  trusting, 
and  very  cooperative.  Low  agreeableness  is 
associated  with  competition. 

Trust 

Morality 

Altruism 

Cooperation 

Modesty 

Sympathy 

Neuroticism 

An  individual  that  feels  emotions  intensely. 

They  often  are  associated  with  anxiety, 
stress,  irritability,  or  sadness.  Predisposed 
to  viewing  the  environment  as  negative  or 
threatening. 

Anxiety 

Anger 

Depression 

Self-consciousness 

Immoderation 

Vulnerability 
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B-3  Motivation 


The  variables  are  identified  and  defined  as  the  following. 

B-3.1  Grit 

A  person’s  ability  to  persist  at  a  lifetime  or  long-term  goal  either  professional  or 
educational  despite  failures  and  challenges  (Duckworth  and  Quinn  2009;  Vallerand 
et  al.  2014). 

B-3.2  Intrinsic  vs.  Extrinsic  Motivation 

Intrinsic  Motivation:  This  is  an  internal  desire  and  drive  to  seek,  know,  accomplish, 
and  stimulate. 

Extrinsic  Motivation:  The  source  of  desire  and  drive  to  seek,  know,  accomplish, 
and  stimulate  comes  from  the  external  environment;  a  reinforcer  to  encourage  an 
individual  to  learn  would  be  an  extrinsic  motivator  (e.g.,  del  Soldato  and  du  Boulay 
1999;  Kember  et  al.  1999;  Noels  et  al.  1999). 

B-3.3  Student  Autonomy 

Locus  of  control  (perceived  student  autonomy):  the  student’s  perception  that  they 
are  able  to  influence  or  control  the  outcome  of  events  (Deci  and  Ryan  1985). 

Control  (actual  student  autonomy):  instructional  strategies  in  which  the  student  is 
given  responsibility  for  learning  (Kember  et  al.  1999),  able  to  make  decisions  (Noel 
et  al.  1999),  and  encouraged  to  make  decisions  (Reeve  and  Deci  1996)  regarding 
their  learning  progress. 

B-3.4  Self-Esteem 

A  person’s  overall  feeling  of  worth  for  themselves  (Baumeister  1995;  Rosenberg 
1979). 

B-3.5  Seif-Efficacy/Competence 

A  person’s  belief  about  their  capability  to  successfully  complete  a  task  to  achieve 
a  goal  (Bandura  1997). 

B-3.6  Goal  Orientation 

There  are  6  types  of  goal  orientation  (Elliot  et  al.  2011): 

•  Task  approach:  Motived  to  do  a  task  correctly 
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•  Task  avoidance:  Avoiding  doing  the  task  incorrectly 

•  Self-  approach:  Doing  better  than  before 

•  Self-avoidance:  Avoiding  doing  worse  than  before 

•  Other-approach:  Doing  better  than  others 

•  Other- avoidance:  Avoiding  doing  worse  than  others 

B-3.7  Student  Approaches  to  Learning  (SAL) 

This  approaches  are  deep,  surface,  and  strategic  (Marton  and  Saljo  1976;  Laurillard 

1979;  Ramsden  1979): 

•  Deep  learners:  Actively  seek  to  understand  the  material/subject.  Interact 
vigorously  with  the  content;  make  use  of  evidence,  inquiry,  and  evaluation; 
relate  new  ideas  to  previous  knowledge;  and  tend  to  read  and  study  beyond 
the  course  requirements.  Motivated  by  interest.  Tend  to  have  higher  grades, 
retain,  and  transfer  information  at  higher  rates  (Biggs  1987). 

•  Surface  learning:  Learn  to  repeat  what  they  have  learned.  Memorize 
information  and  take  a  narrow  view  and  concentrate  on  the  detail.  Fail  to 
distinguish  principles  from  examples  and  are  motivated  by  fear  of  failure. 

•  Strategic  learning:  Seek  to  obtain  a  high  grade.  Organize  time  and  effort  for 
greatest  effect  to  grade.  Only  study  materials  that  appropriate.  Use  previous 
exams  and  assessments  to  predict  questions.  Motivated  by  grades. 

B-3.8  Affect  Predisposition 

This  reflects  sad  versus  happy,  anxious/nervous  versus  calm/relaxed;  pessimistic 

versus  optimistic. 

B-3.9  Reinforcement  Sensitivity  Theory  (RST) 

(Gray  1970;  Gray  and  McNaughton  2000) 

•  Behavioral  Inhibition  System  (BIS):  the  epicenter  of  anxiety  that  is 
activated  by  sources  of  conflict  and  resolution. 

•  Behavioral  Activation  System  (BAS):  mediates  responses  to  conditioned 
signals  of  reward  and  signals  of  relieving  non  punishment. 

•  Fight-Flight-Freeze  System  (FFFS):  mediates  reactions  . 
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B-3.10  Values 


“Enduring  belief  that  a  specific  end-state  of  existence  or  specific  mode  of  conduct 
is  preferred  to  a  different  end-state  or  mode  of  conduct  for  living  one’s  life” 
(Rokeach  1973).  This  is  a  set  of  models  for  attitudes,  beliefs,  and  behavior. 

B-3.11  Task  Interest 

This  would  be  to  gauge  the  student’s  interest  in  the  task  to  be  performed  or  that  was 
just  performed. 

B-3.12  Self-Regulation 

Emotion:  The  student’s  ability  to  maintain  task  focus  and  confidence  in  the  face  of 
criticism  and  difficulty  (Knafer  and  Heggestad  1999). 

Management  of  learning:  the  student’s  ability  to  apply  metacognitive  strategies  to 
plan  their  time,  monitor  progress,  and  ensure  sufficient  effort  is  extended  on  the 
task  (Hattie  et  al.  1996)  and  to  seek  information  when  needed. 

B-3.13  Need  for  Cognition 

An  individual’s  tendency  to  engage  in  and  enjoy  effortful  cognitive  endeavors 
(Cacioppo  et  al.  1984). 
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Intentionally  lelt  blank. 
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Appendix  C.  Prototype 


The  appendix  appears  at  it  was  received  without  editorial  change. 
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Motivation  Assessment 


Directions:  This  motivational  assessment  is  used  for  an  intelligent  tutor.  To  ensure 
a  proper  plan,  please  answer  openly  and  honestly.  There  are  no  right  or  wrong 
answers.  The  first  answer  you  think  of  is  usually  the  best  one.  The  scale  below  is 
used  to  answer  the  questions.  If  the  statement  is  true  to  you,  select  strongly  agree. 
Click  strongly  disagree,  if  it  is  not  true  of  you.  If  the  statement  is  more  or  less  true, 
find  the  words  between  strongly  agree  and  strongly  disagree  that  best  describes  you. 

I  always  work  hard  to  learn  because  learning  is  interesting. 

O  Strongly  disagree  (1) 

O  Disagree  (2) 

O  Somewhat  disagree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  agree  (5) 

O  Agree  (6) 

O  Strongly  agree  (7) 

I  am  interested  in  a  wide  range  of  topics. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  to  feel  in  complete  control  over  my  learning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Creating  new  ideas  and  being  creative  is  the  only  motivation  I  need  to  learn. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

If  I  could  pick  different  pathways  for  learning,  I  would  feel  I  have  more  choice  over 
my  learning. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  gain  interest  in  a  topic  as  I  build  more  knowledge  and  understanding. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Understanding  the  material,  provides  me  the  greatest  satisfaction. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  have  a  hard  time  finding  task  that  are  interesting. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

When  I  am  unable  to  comprehend  the  material,  I  am  capable  of  finding  it  on  my 
own. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  always  put  in  maximum  effort  to  achieve  goals. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  start  out  motivated  but  lose  interest  as  time  goes  on. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  start  out  motivated  but  lose  interest  as  time  goes  on. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  do  not  want  to  work  very  hard  for  this  task.  I  just  want  to  pass. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Learning  and  understanding  the  information  is  more  important  than  the  grade  I 
receive. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

My  goal  when  taking  this  course  is  to  avoid  preforming  poorly. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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My  goal  when  taking  this  course  is  to  avoid  doing  worse  on  exams  than  previously. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

My  goal  when  taking  a  course  is  to  perform  better  on  exams  than  I  have  previously. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

My  goal  when  taking  this  course  is  to  provide  the  correct  answers. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

My  goal  when  taking  a  course  is  to  understand  the  material  as  well  as  getting  the 
questions  correct. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  will  not  give  up  and  will  continue  to  strive  for  understanding  no  matter  the 
complexity  of  the  material. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

When  I  am  tired  of  a  question,  I  will  guess. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Even  if  the  task  is  boring,  I  will  work  hard. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Even  when  I  am  tired  of  a  question,  I  still  persist  for  accuracy. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  like  to  delay  my  work  until  it  is  necessary. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  do  not  review  any  coursework. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  can  pass  by  memorizing  the  information,  even  if  I  do  not  understand  the  material. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  try  hard  so  I  do  not  look  stupid/dumb  compared  to  my  peers. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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When  I  am  unable  to  comprehend  the  material,  I  will  only  study  the  easy  parts. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  pick  activities  that  give  me  instant  progress. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  motivated  for  my  own  satisfaction. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  never  think  beyond  what  the  task  is  asking  me  to  learn. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  do  not  try  to  learn  when  I  am  bored. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  often  miss  important  information  because  my  mind  wanders. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  always  pay  attention  so  I  don't  miss  important  information. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  enjoy  a  task  that  requires  a  lot  of  thinking  and  challenge. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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It  is  my  fault,  if  I  do  not  pass  or  understand  the  information  from  the  task. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  begin  a  task  as  soon  as  its  assigned. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  motivated  due  to  social/family  pressures. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  do  not  like  to  think  about  abstract  concepts. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  prefer  tasks  that  are  hard  as  opposed  to  easy. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  strive  to  be  meticulous  and  pay  attention  to  detail  to  complete  task. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  finish  any  task  that  I  start. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  require  stimulants  to  keep  me  focused  on  a  boring  task. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  always  perform  poorly  on  task  and  test. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Being  able  to  see  small  successes  helps  me  be  more  driven  to  finish. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  confident  I  will  well  on  assignments  and  test  given  to  me. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It's  OK  to  make  a  mistake  because  it  is  just  a  part  of  learning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Thinking  about  mistakes  makes  me  worry. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  often  worry  about  not  meeting  others  standards. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

When  I  am  nervous,  I  have  a  hard  time  performing. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  welcome  feedback  because  it  guides  me  in  learning. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  able  to  push  my  fear  to  the  side  to  begin  a  task. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  become  upset  when  I  answer  a  question  incorrectly. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  don't  worry  about  what  others  think. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  struggle  to  remain  positive. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  always  positive. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  know  I  am  going  to  get  the  answers  right  because  I  have  in  the  past. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

When  I  am  nervous,  I  am  still  able  to  perform  the  task  to  the  best  of  my  ability. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  avoid  complex  task  because  I  may  not  be  successful  of  provide  correct  answers. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  take  feedback  as  negative  criticism  even  when  its  meant  to  be  constructive. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  feel  like  I  have  triggers  that  will  make  me  sad  at  any  moment. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  feel  satisfied  with  my  performance  on  most  tasks. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  usually  am  not  satisfied  by  my  performance  on  a  task  because  I  could  have  done 
better. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  don't  want  other  students  to  think  I  am  smarter  than  them. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  often  compare  myself  to  others. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  must  receive  a  score  that  is  similar  or  better  than  my  peers. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  want  to  be  the  best. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  afraid  to  compare  myself  to  others. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Being  embarrassed  does  not  affect  me. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  compare  myself  to  others  in  an  anonymous  manner. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  always  perform  better  than  my  peers. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  always  try  to  win  at  anything  I  do. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  try  hard  so  I  do  not  look  stupid/dumb  compared  to  my  peers. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  always  try  to  show  all  my  abilities  and  have  peers  admire  my  work. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Being  compared  to  others  is  something  I  detest. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  find  that  in  competitive  scenarios  I  lose  track  of  the  world  around  me. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  consistently  feel  like  I  am  underperforming  compared  to  my  peers. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  would  try  harder  if  I  could  work  toward  a  harder  goal. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  prefer  to  write  my  own  goals. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  would  remember  more  if  my  goal  was  more  difficult. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Reaching  smaller  accomplishments,  helps  me  drive  toward  reaching  my  main  goal. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  tasks  where  I  get  to  figure  out  solutions  to  problems. 

O  Like  a  great  deal  (1) 

O  Like  a  moderate  amount  (2) 

O  Like  a  little  (3) 

O  Neither  like  nor  dislike  (4) 

O  Dislike  a  little  (5) 

O  Dislike  a  moderate  amount  (6) 

O  Dislike  a  great  deal  (7) 

It  is  important  for  me  to  know  why  I  need  to  learn  this  material. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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You  should  do  as  told  and  follow  rules  at  all  times. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  effective  at  managing  my  time. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Helping  me  manage  my  time  will  reduce  "off  task"  behavior. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It  would  help  if  the  tutor  reminded  me  to  remain  focused,  if  too  much  time  passes 
without  answering. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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If  I  do  have  a  question,  I  like  to  find  the  answer  on  my  own. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  do  better  when  guidance  is  provided  only  when  I  ask  for  it. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  understand  and  remember  the  content  better  if  I  am  listening  to  it. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  understand  and  remember  the  content  better  if  I  am  reading. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  understand  and  remember  the  content  better  if  I  watch  a  video. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  understand  and  remember  the  content  if  it  remains  at  a  complex  level. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  understand  and  remember  the  material  better  if  it  is  at  an  easier  level. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  only  need  to  read,  watch,  or  listen  to  the  material  once  in  order  to  remember  and 
understand  the  material. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  will  understand  and  remember  the  material  if  I  listen,  read,  or  watch  it  several 
times. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  understand  and  remember  the  material  better  if  I  take  notes. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  can  remember  content  better  if  I  take  notes  with  a  paper  and  pencil. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  to  create  my  own  outline  to  understand  the  information. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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A  premade  outline  that  I  need  to  fill  in  helps  me  learn  the  information  and  remain 
focused. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  premade  outline  helps  me  be  more  successful  but  not  understand  the 
information. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

When  reading,  listening,  or  watching  I  automatically  pick  out  the  important  points 
and  details. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It  would  help  me  to  have  the  tutor  tell  me  the  main  points  and  details  from  my 
learning. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  reflect  on  my  learning  for  every  task. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  more  attentive  when  I  feel  threatened. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  more  attentive  when  I  feel  satisfaction. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  to  participate  actively  in  order  to  understand  the  material. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  not  motivated  when  I  am  forced  to  complete  all  the  practice  questions,  even 
though  I  have  been  very  accurate  in  answering. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It  would  be  motivating  if  I  could  practice  until  I  felt  I  understood  the  information. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  to  be  prompted  to  reflect  on  my  learning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  don't  need  reviews  because  I  constantly  test  myself  until  I  make  sure  I  completely 
understand  the  content. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  need  the  information  to  be  given  to  me  all  at  once. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It  would  be  best  if  I  could  control  the  amount  of  information  given  to  me. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  checklist/plan  would  be  beneficial  to  me. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  make  my  own  checklist/plan. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  do  not  like  checklists  because  I  already  have  a  plan  in  my  head. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Providing  the  assessment  questions  first  allows  me  to  learn  at  a  deeper  level. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  capable  of  deciding  not  to  take  a  pre-assessment  because  it  would  waste  my 
time  if  I  was  not  knowledgeable  on  the  subject. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  know  my  level  of  understanding  on  the  topic.  I  would  like  the  choice  to  take  a 
pre-assessment  if  I  am  knowledgeable  to  individualize  my  learning  plan. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  would  work  harder  if  a  pre-assessment  let  me  skip  the  parts  I  already  know. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  low  score  on  a  pre-assessment  would  cause  me  to  provide  less  effort. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  prefer  to  work  in  a  group. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  learn  on  my  own. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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A  sequential  flow  of  the  content  gives  me  confidence  that  I  will  be  able  to  learn  the 
material. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Allowing  me  to  choose  the  flow  of  the  content  motivates  me  to  learn. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  do  better  when  guidance  is  provided  only  when  I  ask  for  it. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  have  a  hard  time  remembering  information  when  I'm  learning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  always  perform  poorly  on  tasks  and  tests. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  feel  very  tense  reading  complex  material. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  can  easily  comprehend  material. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  would  learn  more  if  the  tutor  did  not  provide  me  the  answer  once  I  got  it  wrong. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  would  learn  more  if  the  tutor  gave  me  the  correct  answer  right  away. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  would  learn  more  with  guidance,  but  may  try  harder  if  I  knew  I  would  receive 
less  points  for  the  hints. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  would  learn  more  if  I  had  to  provide  a  reason  to  support  my  answer  choice  for 
multiple  choice. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It  would  be  beneficial  if  the  tutor  told  me  to  look  for  the  answer  before  telling  me 
the  answer. 

□  Strongly  agree  (1) 

□  Agree  (2) 

□  Somewhat  agree  (3) 

□  Neither  agree  nor  disagree  (4) 

□  Somewhat  disagree  (5) 

□  Disagree  (6) 

□  Strongly  disagree  (7) 
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If  I  answer  a  question  incorrectly,  it  would  be  beneficial  for  the  tutor  to  provide  me 
with  additional  information  on  the  topic. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  work  faster  if  I  know  I  am  under  a  time  constraint. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  learn  more  when  I  am  assessed  at  the  end  of  a  task. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  learn  better  if  I  have  to  explain  my  answers  as  opposed  to  selecting  an  answer  in 
a  multiple  choice  format. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  annoyed  with  too  many  multiple  choice  questions. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  learn  more  when  I  do  not  know  what  I  will  be  assessed  on. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  enjoy  continuously  being  assessed  to  obtain  immediate  feedback. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Seeing  questions  before  the  task  limits  the  amount  of  information  I  leam. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  should  be  given  a  warning  to  improve  if  I  am  not  maintaining  full  effort. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  find  it  important  to  have  fun  during  the  task. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

If  I  provided  the  wrong  answer,  it  would  be  beneficial  for  the  tutor  to  re-ask  a 
similar  question  later. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Sometimes  I  am  overwhelmed  by  the  amount  of  information  I  have  to  read. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Aesthetics  help  motivate  me. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  find  that  I  perform  better  when  I  have  breaks. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Knowing  I  am  close  to  finishing  an  assessment  makes  me  try  harder. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Organization  and  structure  are  very  important  to  me. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  more  motivated  to  learn  if  I  have  to  teach  someone  else. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Reading  complex  material  excites  me. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  don't  review  any  coursework. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  want  to  pick  my  own  level  of  complexity,  knowing  I  can  change  the  level  at  any 
moment. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  would  prefer  to  let  the  tutor  decide  the  complexity  level  I  need. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Knowing  I  will  receive  a  reward  for  reaching  a  goal  helps  me  remain  focused  and 
provide  effort. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  to  be  praised  often  to  feel  valued. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Surprising  or  unexpected  rewards  motivate  me. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  love  when  the  instructor  uses  me  as  an  example  to  others. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  motivated  by  acknowledgment  from  friends  and  family. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  motivated  by  acknowledgment  from  my  team  mates. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  motivated  by  acknowledgment  from  those  that  have  a  higher  authority  than 
myself. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  would  do  better  if  learning  turned  into  a  game  and  competition. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

My  image  is  important  to  me. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  would  perform  better  if  I  was  anonymously  compared  to  others. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

It  is  enjoyable  to  show  my  teacher  I  am  more  knowledgeable  than  my  classmates. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  enjoy  sharing  my  grades/score  with  others. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  enjoy  keeping  my  grades/score  private. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  to  have  a  break  at  times  to  help  keep  focused. 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  need  a  break  after  30  minutes  of  work. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  need  a  break  after  an  hour  of  work. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  don't  need  any  breaks. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

My  satisfaction  increases  when  I  receive  points  and  feedback  on  my 
accomplishments . 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  would  rather  not  receive  a  reward. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  like  to  see  my  progress  in  reaching  a  goal  (progress  bar)  to  remain  motivated. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  try  harder  if  I  know  the  grade  will  be  seen  by  everyone. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  try  harder  if  I  know  that  I  cannot  retake  the  class  for  a  better  grade. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  will  work  harder  if  I  have  to  start  over  if  I  provide  too  many  incorrect  answers. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  am  motivated  to  improve  my  performance  when  I  receive  a  negative  consequence 
for  poor  performance. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  should  be  given  a  warning  to  improve  if  I  am  not  maintaining  full  effort. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  celebrate  my  accomplishments. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  am  motivated  when  I  have  challenges  that  are  high  risk  but  also  high  reward. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  like  to  be  recognized  quietly. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  like  to  be  recognized  in  front  of  others. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Rewards  need  to  be  proportional  to  my  level  of  effort  or  they  are  not  worth  my 
effort. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Rewards  that  are  valuable  to  me  change  depending  on  the  context. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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I  take  risk  even  when  there  is  a  potential  negative  consequence. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

I  avoid  task  risks. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

What  else  motivates  you? 

What  strategy  provided  to  you  would  influence  your  effort? 

If  you  could  add  one  thing  to  learning  to  keep  you  motivated,  what  would  it  be? 

What  is  the  number  one  thing  that  prevents  you  from  being  more  motivated? 

Does  this  item  make  you  want  to  work  harder  in  order  to  earn?  Select,  strongly 
agree  if  you  would  work  your  hardest  for  that  item.  Select,  strongly  disagree  if  you 
would  not  but  forth  any  more  effort  in  your  work  for  that  item.  Select  the  item  in 
the  middle  that  hold  true  to  how  much  harder  you  would  work  to  earn. 

Electronic  badges  given  through  the  tutor  as  you  progress  through  your  work. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Achievements  provided  with  points  for  accomplishing  something  (e.g.  Speed, 
accuracy,  or  so  many  correct  in  a  row) 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Receiving  more  positive  feedback  than  negative. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Praise  regarding  my  effort  instead  of  how  much  I  know. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Praise  on  my  knowledge  instead  of  praise  on  my  effort. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Praise  on  both  knowledge  and  effort. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Receiving  points  throughout  the  learning  process  instead  of  receiving  grades  at  the 
end. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Having  a  progress  bar  that  shows  my  performance  relative  to  my  goal. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Having  the  opportunity  to  pick  the  type  of  assignment  (written,  multiple  choice, 
oral,  or  a  game). 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Working  to  get  points  as  a  team  to  boost  your  score  (e.g.,  If  all  team  members  have 
an  A  then  you  get  a  bonus). 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Playing  a  beat  the  clock  game. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

An  email  or  test  sent  to  my  phone  recognizing  my  efforts  and  accomplishments. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Playing  a  digital  slot  machine  for  accomplishments. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Having  an  avatar  (person)  that  I  can  buy  items  for  with  my  points. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Owning  an  army,  pet,  etc.  that  I  look  after  and  protect  with  points  earned. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Independent  free  time. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Having  an  animated  clip  in  the  middle  of  learning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Having  an  animated  clip  in  the  beginning  to  pump  me  up. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Having  an  animated  clip  at  the  end  for  a  mini  celebration. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Taking  a  break  with  motivational  music. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

having  motivational  music  at  the  beginning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Being  able  to  listen  to  music  while  I  learn. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Motivational  quotes  to  help  me  persist. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  mystery  jackpot  (e.g.,  your  name  is  pulled  out  of  a  drawing  for  meeting  a  goal). 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Providing  a  donation  to  a  charity. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Have  free  time  to  volunteer. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Spending  time  with  your  friends  or  family. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

The  opportunity  to  help  tutor  someone  about  the  knowledge  you  successfully 
learned. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Watch  a  sports  game. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Having  a  longer  lunch  than  normal. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Leave  work  early. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Sleep  in  and  arrive  to  work/school  later. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Have  a  paid  day  off. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Play  a  board  game. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Play  a  video  game. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Watch  a  movie. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Spending  extra  time  with  your  family. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Extra  time  to  hang  out  with  your  classmates  or  team. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Time  to  hang  out  with  your  friends. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Dancing 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Singing 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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A  special  parking  spot. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Having  someone  do  your  dishes,  laundry,  or  cleaning. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Reading  a  book. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Spending  time  with  someone  in  a  higher  command  to  learn  from  them. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Playing  or  watching  football. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Fishing. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Use  of  a  fitness  center. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Practicing  at  a  shooting  range. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Time  to  work  out. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Time  to  do  an  outdoor  adventure  (e.g.  climbing,  biking,  camping,  or  skydiving) 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Going  to  see  a  theater  play  or  music  concert. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Creativity  time  with  art. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 


Approved  for  public  release;  distribution  is  unlimited. 

178 


A  photography  class  or  time  to  take  pictures. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Swimming. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Time  to  work  in  the  garden. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Play  a  round  of  golf. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Boating. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Lifting  weights. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Playing  tennis  or  racquetball. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Go  off-roading 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Being  able  to  run  errands  in  the  middle  of  the  day. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Having  your  hair  done. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Go  shopping. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Getting  a  manicure  or  pedicure. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Raising  your  chances  of  a  promotion. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  trip  to  go  visit  a  friend/family  who  lives  far  away. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Going  to  a  theme  park. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Hiking. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 


Approved  for  public  release;  distribution  is  unlimited. 

182 


Thrill  seeking  adventures  such  as  bungee  jumping,  scuba  diving,  skydiving  etc. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Going  to  see  a  concert. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  relaxation  room 
O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Playing  an  instrument 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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A  massage. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  certificate  of  completion  for  my  portfolio. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

A  gift  card. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Extra  pay  on  a  check  or  bonus. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Receiving  money. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Receiving  small  gifts. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Free  beverage. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Free  food. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Receiving  free  coffee  or  tea. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Receiving  a  hug. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Subscription  to  a  magazine. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Renting  a  sports  car. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 
O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Have  a  teammate  cook  dinner  for  you. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q293  Physical  badges  to  show  your  accomplishments. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q294  Subsidized  childcare. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q295  Losing  points  because  you  answered  incorrectly. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Q296  Losing  points  because  you  answered  multiple  questions  incorrectly. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q297  Loss  of  points  from  failing. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q298  Starting  the  course  over  because  you  had  too  many  questions  wrong. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q299  Receiving  acknowledgments  from  each  class  (e.g.  highest  score,  most 
improved). 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 


Approved  for  public  release;  distribution  is  unlimited. 

188 


Q300  In  a  meeting,  be  acknowledged  for  your  work. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q301  My  name  on  an  honor  roll. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q302  Being  able  to  search  for  others  scores. 

O  S  trongly  agree  ( 1 ) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q303  Having  your  achievements  hung  on  the  wall. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 
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Q304  Award  ceremonies. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q305  A  leader  board  that  compares  you  to  others. 

O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Q306  Allowing  others  to  provide  comments  and  likes  on  your  answers  and  effort. 
O  Strongly  agree  (1) 

O  Agree  (2) 

O  Somewhat  agree  (3) 

O  Neither  agree  nor  disagree  (4) 

O  Somewhat  disagree  (5) 

O  Disagree  (6) 

O  Strongly  disagree  (7) 

Are  there  other  reinforcers  that  would  make  you  work  harder?  Please  list  them. 

You  have  completed  all  the  questionnaires.  Your  responses  will  enable  us  to 
understand  how  individuals  are  motivated.  Thank  you  for  participating! 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


ABA 

Applied  behavior  analysis 

AR 

augmented  reality 

ARCS 

Attention,  Relevance,  Confidence,  and  Satisfaction  Model 

ARL 

US  Army  Research  Laboratory 

ASA 

attraction-selection-attrition 

BIP 

behavioral  intervention  plans 

BAS 

behavioral  avoidance  scale 

BIS 

behavioral  inhibition  scale 

CAMBRA 

Causal  Analytical  Modeling  via  Blocked  Regression 
Analysis 

CBT 

Computer  Based  Tutoring 

DA 

Dopamine 

DRL 

Differential  reinforcement  rate  of  low  rates 

DRRPI 

Dunn-Rankin  Reward  Preference  Inventory 

EBR 

eye  blink  rate 

ECG 

electrocardiogram 

EF 

elaborated  feedback 

EPQ-R  Short  Form 

Eysenck  Personality  Questionnaire-  Revised  Short  Form 

FBA 

functional  behavioral  assessments 

fMRI 

functional  magnetic  resonance  imaging 

GIFT 

Generalized  Intelligent  Framework  for  Tutoring 

HR 

heart  rate 

IDs 

Instructional  Designers 

I-E-0 

input-environment-outcome 

IMI 

Intrinsic  Motivation  Inventory 
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IMMS 

Instructional  Materials  Motivation  Survey 

IPIP 

International  Personality  Item  Pool 

IRB 

Internal  Review  Board 

ISD 

Instructional  System  Design 

ISD 

Instructional  Systems  Design 

1ST,  UCF 

Institute  for  Simulation  and  Training  at  the  University  of 
Central  Florida 

ITS 

Intelligent  Tutoring  System 

KCR 

knowledge  of  correct  response 

KR 

knowledge  of  results 

LRS 

Feamer  Record  Store 

LTLM 

Fong  Term  Feamer  Model 

LTP 

long  term  potentiation 

MAT 

Motivator  Assessment  Tool 

MLSQ 

Motivated  Feaming  Strategies  Questionnaire 

RR-PVQ 

Revised  and  Refined-  Portrait  Values  Questionnaire 

MOOC 

Massive  Open  Online  Course 

MSLQ 

Motivated  Strategies  for  Fearning  Questionnaire 

MTL 

medial  temporal  lobe 

MTURK 

Amazon  Mechanical  Turk 

NAc 

nucleus  accumbens 

NEO-FFI 

NEO  Five  Factor  Inventory 

OIT 

Organismic  Integration  Theory 

OMQ 

Online-Motivation  Questionnaire 

PAFS 

Patterns  of  Adaptive  Feaming  Survey 

PBS 

Positive  Behavioral  Supports 

PENS 

Player  Experience  of  Need  Satisfaction 
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PFC 

prefrontal  cortex 

RIMMS 

Reduced  Instructional  Materials  Motivation  Survey 

RL 

reward  related  process 

RPE 

Reward  prediction  error 

SAL 

Student  Approaches  to  Learning 

SDT 

self-determination  theory 

SRL 

Self-regulated  Learning 

TIPI 

Ten  Item  Personality  measure 

VR 

virtual  reality 

VTA 

ventral  tegmental  area 
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(PDF)  INFORMATION  CTR 

DTIC  OCA 
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1  ARL 
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8 

(PDF) 


ARL 

SFC  PAUL  RAY  SMITH 
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RDRL  HRO 
RDRL  HRF 
RDRL  HRA 
RDRL  HRR 
RDRL  HRA  C 


COL  H  BUHL 
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I  MARTINEZ 
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D  HEADLEY 
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A  DECOSTANZA 
RDRL  HRF  B 
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RDRL  HRA  B 
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